WO2022121186A1 - Method and apparatus for routing inspection of oil and gas pipelines on the basis of target matching - Google Patents

Abstract

A method and an apparatus for routing inspection of oil and gas pipelines on the basis of target matching, a computer device and a computer-readable storage medium. The method comprising: extracting a target image from an aerial image in an oil and gas pipeline region; performing image feature extraction on the target image by using a neural network model, to obtain a first image feature, wherein the neural network model is obtained in advance by performing training on the basis of a plurality of reference image samples, and the reference image samples are labeled with compliant target images, and the neural network model performing image feature extraction on the compliant target images in the training process, to obtain a second image feature; determining whether the second image feature corresponding to a detected compliant target image matches the first image feature; and if not, determining that the target image is an anomalous target image. The method increases the efficiency of identifying a violated target in an oil and gas pipeline region.

Description

Oil and gas pipeline inspection method, device, equipment and medium based on target matching

This application claims the priority of the Chinese patent application with the application number of 2020114612776 and the title of the invention, which was filed with the China Patent Office on December 11, 2020, and whose name is "target matching-based oil and gas pipeline inspection method, device, equipment and medium", all of which The contents are incorporated herein by reference.

technical field

The present application relates to the field of artificial intelligence, and in particular, to an oil and gas pipeline inspection method based on target matching, an oil and gas pipeline inspection device based on target matching, computer equipment, and a computer-readable storage medium.

Background technique

If there is illegal construction above the ground in the oil and gas pipeline area, the pipeline may be damaged. Therefore, drones are generally used to monitor oil and gas pipelines in the process of cruising, and to identify illegal construction projects in the pipeline area and construction projects that have been registered in the warehouse.

At present, the main detection and identification method for illegal buildings is to extract the buildings in the image by processing the remote sensing images taken by drones at high altitudes through simple corner detection, line segmentation and other traditional image processing methods. out, and then through manual discriminant analysis to confirm whether it is an illegal building.

However, the inventor realized that, in this way, although the traditional image processing method can be used to detect the building from the aerial image, it still requires a lot of manual work to determine whether the building belongs to the illegal building other than the existing buildings in the pipeline area. Judgment, which requires a lot of labor cost investment, and reduces the efficiency of inspection of illegal targets in the oil and gas pipeline area.

The above content is only used to assist the understanding of the technical solutions of the present application, and does not mean that the above content is the prior art.

technical problem

The main purpose of the present application is to provide an oil and gas pipeline inspection method based on target matching, an oil and gas pipeline inspection device based on target matching, computer equipment and a computer-readable storage medium, aiming to solve how to inspect oil and gas pipelines in the process of inspection. , the problem of improving the efficiency of identifying offending targets within the oil and gas pipeline area.

technical solutions

In order to achieve the above-mentioned purpose, the present application provides an oil and gas pipeline inspection method based on target matching, comprising the following steps:

Use drones to collect aerial images in the oil and gas pipeline area;

extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

Use a neural network model to perform image feature extraction on the target image to obtain the first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliant target images; The neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

According to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

If so, determine that the target image is the compliance target image;

If not, it is determined that the target image is an abnormal target image.

In order to achieve the above object, the present application also provides an oil and gas pipeline inspection device based on target matching, and the oil and gas pipeline inspection device based on target matching includes:

The acquisition module is used to collect aerial images in the oil and gas pipeline area by using drones;

a processing module for extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

The feature extraction module is used for extracting image features of the target image by using a neural network model to obtain a first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with a compliance target image; the neural network model performs image feature extraction on the compliance target image during training to obtain a second image feature;

a detection module, configured to detect a compliance target image that satisfies a preset condition according to the second position and the first position corresponding to the compliance target in the compliance target image;

a judgment module for judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

a first determination module, configured to determine if the target image is the compliance target image;

The second determination module is configured to determine that the target image is an abnormal target image if not.

To achieve the above purpose, the present application also provides a computer device, the computer device comprising:

The computer equipment includes a memory, a processor, and a target matching-based oil and gas pipeline inspection program that is stored on the memory and can run on the processor, and the target matching-based oil and gas pipeline inspection program is described When the processor executes, the oil and gas pipeline inspection method based on target matching is realized;

Wherein, the steps of the oil and gas pipeline inspection method based on target matching include:

Use drones to collect aerial images in the oil and gas pipeline area;

extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

Use a neural network model to perform image feature extraction on the target image to obtain the first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliant target images; The neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

According to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

If so, determine that the target image is the compliance target image;

If not, it is determined that the target image is an abnormal target image.

In order to achieve the above object, the present application also provides a computer-readable storage medium, on which is stored an oil and gas pipeline inspection program based on target matching, and the oil and gas pipeline inspection program based on target matching is processed. The oil and gas pipeline inspection method based on target matching is realized when the controller is executed;

Wherein, the steps of the oil and gas pipeline inspection method based on target matching include:

Use drones to collect aerial images in the oil and gas pipeline area;

extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

Use a neural network model to perform image feature extraction on the target image to obtain the first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliant target images; The neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

According to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

If so, determine that the target image is the compliance target image;

If not, it is determined that the target image is an abnormal target image.

beneficial effect

The oil and gas pipeline inspection method based on target matching, the oil and gas pipeline inspection device based on target matching, the computer equipment and the computer-readable storage medium provided by this application utilize artificial intelligence and target detection technology to automatically detect oil and gas collected by unmanned aerial vehicles. The image in the pipeline area is subjected to image recognition and feature extraction to obtain the target building in the image, and the image features corresponding to the compliant buildings that are close to the target building are used to perform feature matching with the image features corresponding to the target building. Based on this, it is judged whether the target building is a compliant building, and if not, it is an illegal building, thereby improving the efficiency of identifying illegal targets in the oil and gas pipeline area during the process of patrolling the oil and gas pipeline area.

Description of drawings

1 is a schematic diagram of steps of an oil and gas pipeline inspection method based on target matching in an embodiment of the application;

2 is a schematic block diagram of an oil and gas pipeline inspection device based on target matching according to an embodiment of the application;

FIG. 3 is a schematic structural block diagram of a computer device according to an embodiment of the present application.

The realization, functional characteristics and advantages of the purpose of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments.

BEST MODE FOR CARRYING OUT THE INVENTION

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

1, in one embodiment, the oil and gas pipeline inspection method based on target matching includes:

Step S10, using unmanned aerial vehicle to collect aerial photography images in the oil and gas pipeline area;

Step S20, extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

Step S30, using a neural network model to perform image feature extraction on the target image to obtain a first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliance targets an image; the neural network model performs image feature extraction on the compliance target image during training to obtain a second image feature;

Step S40, according to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

Step S50, judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

Step S60, if yes, then determine that the target image is the compliance target image;

Step S70, if no, determine that the target image is an abnormal target image.

In this embodiment, the terminal of the embodiment may be a computer device (such as a central data platform), or may be an oil and gas pipeline inspection device based on target matching.

As described in step S10: the drone is an unmanned aircraft, and an image acquisition device is installed on the drone. The terminal establishes a communication connection with the UAV or the data collection site responsible for maintaining the UAV, and the terminal can issue the image acquisition instruction for the oil and gas pipeline area within the designated area to the UAV, so as to control the UAV's monitoring of oil and gas. The area along the pipeline is routinely inspected, and aerial images in the oil and gas pipeline area are collected in real time or regularly.

Optionally, in the process of using the drone to collect aerial images in the oil and gas pipeline area, the drone can also use satellite positioning technology or network positioning technology to determine the longitude and latitude coordinates when the drone currently collects the aerial image, and according to The latitude and longitude coordinates generate position information corresponding to the currently collected aerial image. Among them, the satellite positioning can use the Beidou positioning system or the GPS (Global Positioning System) system.

Optionally, the drone may send the collected aerial image and the location information corresponding to the aerial image to the terminal in real time; or, the drone first stores the collected aerial image and the location information corresponding to the aerial image, and waits for it. When the communication conditions are met after the drone returns to flight (for example, the drone returns to the data acquisition base station), the stored data is sent to the terminal.

As described in step S20: after the terminal receives the aerial image in the oil and gas pipeline area based on the drone, it can use image recognition technology to identify the preset target in the aerial image, and extract the preset target in the aerial image. The image area is developed to generate the target image.

Among them, the preset target can be a building, or other objects except oil and gas pipelines and the surface. The following description takes the target image as a building image as an example (that is, the target in the target image is a building).

Further, the terminal may directly use the latitude and longitude coordinates in the position information corresponding to the aerial image where the target image is located as the first position corresponding to the target in the target image (ie, the real position of the target in the real physical world).

It should be understood that a target image has at least one target (ie, a building); and if no building is detected in the aerial image, there is no need to generate a target image based on the aerial image.

Alternatively, the terminal can also use the latitude and longitude coordinates in the location information corresponding to the aerial image as the midpoint coordinates of the aerial image, and determine the relative positional relationship between the target corresponding to the target image and the midpoint of the aerial image, and collect data according to the drone image. The flying height of the aerial image and the focal length of the image acquisition device determine the scale of the aerial image, and then according to the midpoint coordinates of the aerial image, the relative positional relationship, and the scale of the aerial image, jointly determine the first image corresponding to the target in the target image. a location. In this way, the accuracy of recognizing the target position in the target image can be improved.

Alternatively, the terminal can use the target detector of the pre-trained neural network model to detect and extract the buildings in the aerial image, so as to identify and segment the target image in the aerial image, and assign the target image corresponding to the aerial image where the target image is located. The latitude and longitude coordinates in the position information are used as the position corresponding to the target in the target image, and are recorded as the first position.

As described in step S30: the terminal constructs a neural network model based on the deep convolutional neural network in advance, and the terminal pre-inputs multiple reference image samples into the neural network model for multiple iterative training, until the model converges, and the training completed Neural network model. It should be noted that the number of the reference image samples is large enough, for example, 10,000 samples.

Optionally, the reference image used to generate the reference image sample may be obtained by the terminal using a drone to collect image data of buildings in the area along the oil and gas pipeline in advance, or the terminal may use other image acquisition means to photograph the area along the oil and gas pipeline. obtained from the buildings inside. In the process of collecting the reference image, the terminal will simultaneously record the position information corresponding to the collected reference image, and when generating the corresponding reference image sample based on the reference image, compare the position information corresponding to the reference image with the generated reference image sample. association.

Optionally, the reference image sample is marked with a compliance target image, and the compliance target is represented as a compliant, non-illegal building. The reference image sample may be a compliant target image marked in the sample by a relevant engineer, and based on the location information associated with the reference image sample, the position of the compliant target image (referred to as the second position) is marked. Then the engineer uses the labeled benchmark image samples as the training samples of the neural network model, and inputs the benchmark image samples into the neural network model for training.

Optionally, the target detector of the neural network model is constructed based on the Mask RCNN network of ResNet50, and the Mask RCNN network can be used to identify and segment the target area in the image. The trained neural network model can also be used to perform step S20 to extract the target image from the aerial image and determine the first position corresponding to the target in the target image, that is, the terminal can directly combine the aerial image with the aerial image. The location information associated with the image is input into the neural network model after training. The Mask RCNN network can obtain the global feature map through the convolutional layer of the neural network model, and on this basis, use the region proposal network to extract the region of interest (RoI, Region of Interest) as the target area, and then use the RoI Align method for all target areas. (with at least one area) area extraction and alignment, and normalization processing, and finally in the fully connected layer of the neural network model, the target category (here is the building category) and the target position (the position is based on the aerial image. After the associated position information is obtained), the detection of the target image in the aerial image is realized, and the first position of the target in the target image is obtained.

Optionally, in the process of training the neural network model with the Mask RCNN network, the training of relevant model parameters of the neural network model is performed based on a plurality of benchmark image samples marked with compliant target images, and the back-propagation algorithm is used in the training. .

The basic principle of the back-propagation algorithm is to optimize the network parameters through the gradient descent method with the goal of minimizing the loss function of the training. The formula for gradient descent is:

Among them, θ represents the network parameters, L(θ) is the loss function used for network training, and α is the iterative step size.

For object detection tasks (ie, detecting object regions in images), the loss function for a single prediction result consists of the first loss function L _cls for object classification and the second loss function L _loc for location regression:

where u is the true class of the target (in this case, a building), and p _u is the confidence score that the network predicts that the target is u. t _i and v _i are the real and predicted positions of the target, respectively. in

The influence of abnormal sample points on the back-propagation process can be suppressed, and the formula is:

Optionally, for the reference image sample, during the training of the neural network model, the Mask RCNN network is used to detect and extract the area of the building image in the reference image sample to obtain the compliant target image, and the compliant target image is processed. The image feature extraction is performed to obtain the image feature corresponding to the compliance target image (referred to as the second image feature). At the same time, according to the position information associated with the reference image sample to which the compliance target image belongs, the real position of the compliance target in the compliance target image in the real physical world (referred to as the second position) is determined. It should be noted that the specific technical means used for determining the second position may be the same as the technical means used for determining the first position.

It should be understood that the extraction of image features by using the deep convolutional neural network is realized based on the convolution operation on the image. The image convolution operation is mainly by setting various feature extraction filter matrices (convolution kernels, such as setting a matrix of size 3*3, or 5*5), and then using the convolution kernel in the original image matrix (image It is actually a matrix composed of pixel values) 'sliding' to realize the convolution operation.

Further, the terminal records all the second image features into the database by numbering, and associates the second position corresponding to the compliance target in the compliance target image corresponding to the second image feature with the second image feature.

Optionally, based on the trained neural network model, it can also be used to extract image features of the target image, and the terminal acquires the image features of the target image extracted by the neural network model as the first image features.

As described in step S40: when the terminal obtains the first position corresponding to the target in the target image, it detects whether the second position stored in the database (that is, the second position corresponding to the compliance target in the compliance target image) is There is a second location that satisfies the preset condition.

Optionally, the preset condition includes any one of the following: the second position is within a preset area constructed based on the first position; the distance between the second position and the first position is less than or equal to the preset distance.

Wherein, the terminal may construct a circular area as the preset area with the first position as the midpoint and the preset length as the radius. Of course, the terminal may also take the first position as the midpoint of the preset rectangular area, and construct the preset area based on this. The preset length and the side length of the preset rectangle can be set according to actual needs, which are not limited in this embodiment.

Then the terminal further detects whether there is a second position located within the preset area; if so, it is determined that the compliance target image to which the compliance target corresponding to the second position belongs satisfies the preset condition; A compliant target image for preset conditions.

Optionally, the terminal may also first determine the distance between the first position and each second position, and further detect whether the distance between the first position and the second position is less than or equal to the preset distance; The compliance target image to which the compliance target corresponding to the second position belongs satisfies the preset condition; if not, it is determined that the compliance target image to which the compliance target corresponding to the second position belongs does not meet the preset condition, and if all If none of the compliant target images meet the preset conditions, it is further determined that no compliant target images meeting the preset conditions are detected. The preset distance may be set according to actual needs, which is not limited in this embodiment.

It should be understood that, when the preset distance takes a value of 0, in the compliance target image satisfying the preset condition, the second position corresponding to the compliance target in the figure needs to coincide with the first position.

Optionally, when the terminal detects that there is a compliant target image that satisfies the preset condition, it continues to perform step S50; when the terminal does not detect that there is a compliant target image that meets the preset condition, it directly determines that the target image is abnormal. target image. Among them, the abnormal target in the abnormal target image can be defined as an illegal building or illegal building.

It should be understood that the preset condition is essentially used to measure the distance relationship between the first position and the second position, that is, the compliant target image that meets the preset condition is between the corresponding second position and the first position. distance is closer. In this way, step S50 is substantially performed only based on the compliance target image whose relative position is relatively close to the target image, so as to further determine whether the detected second image feature corresponding to the compliance target image is the same as the first image feature Matching, and do not use the compliant target image whose relative position is far from the target image to perform step S50, so as to avoid using unnecessary image features for image matching, thereby improving the accuracy of subsequent matching image features (because the distance is farther than the matching target image. Even if the image features of the regulated target image match the target image, they cannot be the same building or the same area); and based on the execution of step S40, compliance targets that are unlikely to match the target image can also be screened out. Therefore, the algorithm complexity of subsequent detection of compliant target images matching the target image is reduced, and the efficiency of image matching is improved, thereby improving the efficiency of identifying illegal targets in the oil and gas pipeline area.

Moreover, when the terminal does not detect a compliant target image that satisfies the preset conditions (that is, when the second position corresponding to the compliant target is not detected near the first position corresponding to the target of the target image), it indicates that the target image was located at the previous location. If there is no compliance target set in the area, then the target (ie building) that appears in the area at this time is likely to be the illegal target, so the terminal directly determines the target image as an abnormal target image, thereby reducing the follow-up The algorithm processing of image feature matching improves the efficiency of identifying illegal targets in the oil and gas pipeline area to a certain extent.

As described in step S50: when the terminal detects that there is a compliance target image that satisfies the preset condition (that is, detects that there is a second position that satisfies the preset condition), the terminal acquires the second position corresponding to the compliance target image from the database. Image features (that is, acquiring a second image feature associated with a second position that satisfies a preset condition), and then detecting whether the acquired second image feature matches the first image feature.

Optionally, the terminal detects whether the second image feature is consistent with the first image feature; if so, determines that the second image feature matches the first image feature; if not, determines that the second image feature does not match the first image feature .

Optionally, the terminal may also perform similarity detection on the first image feature and the second image feature, and further detect whether the similarity between the two is greater than or equal to the preset similarity; One image feature matches; if not, it is determined that the second image feature does not match the first image feature. The value range of the preset similarity may be 90%-100%, and if the preset similarity is 100%, it is equivalent to detecting whether the second image feature is consistent with the first image feature.

As described in step S60: when the terminal determines that the detected second image feature corresponding to the compliance target image matches the first image feature of the target image, it indicates that the building in the target image is an existing record in the library If the target image is compliant, the terminal further determines that the target image is a compliant target image.

As described in step S70: when the terminal determines that the detected second image feature corresponding to the compliance target image does not match the first image feature of the target image, it means that the building in the target image is not pre-recorded in the library is an abnormal target, the terminal further determines that the target image is an abnormal target image, and determines that the target in the target image is an abnormal target (ie, an illegal building or an illegal building).

It should be understood that, based on reverse thinking, the reference image samples used to train the neural network model can also be generated based on abnormal target images. In this case, when it is detected that the second image feature matches the first image feature, the target image is determined. The image is an abnormal target image, and if the second image feature does not match the first image feature, it is determined that the target image is a compliant target image. However, generally speaking, it is easier to collect compliant target images than abnormal target images in the oil and gas pipeline area, that is, it is easier to collect and generate benchmark image samples based on compliant target images, thereby improving the efficiency of building and training neural network models. Generate benchmark image samples based on compliant target images.

In one embodiment, artificial intelligence and target detection technology are used to automatically perform image recognition and feature extraction on images in the oil and gas pipeline area collected by unmanned aerial vehicles to obtain the target building in the image, and use the location of the target building to match the target building. The image features corresponding to similar compliant buildings are matched with the image features corresponding to the target building, and based on this, it is judged whether the target building is a compliant building, and if not, it is a violating building, thus improving the inspection process. Efficiency in identifying offending targets within the oil and gas pipeline area during the process of the oil and gas pipeline area.

In an embodiment, based on the above-mentioned embodiment, the step of judging whether the second image feature corresponding to the detected compliance target image matches the first image feature includes:

Step S51, obtaining the first hash code of the first image feature, and obtaining the second hash code of the second image feature corresponding to the compliance target image obtained by detection;

Step S52, determine the Hamming distance between the first hash code and the second hash code;

Step S53, using the Hamming distance to determine whether the second image feature matches the first image feature.

In this embodiment, when the terminal detects that there is at least one compliance target image that satisfies the preset condition, the terminal acquires the second image feature corresponding to the compliance target image that satisfies the preset condition from the database.

Optionally, in this embodiment, an image retrieval algorithm based on a hash function is adopted, and corresponding hash codes are obtained by substituting image features into the hash function set. Among them, the hash function set is:

H(x)=h ₁ (x),h ₂ (x),...,h _K (x)

Among them, h is the hash function, the hash function set H is composed of multiple hash functions, and K is the number of hash functions. For the image features x _i (i=1, 2, .

Optionally, the terminal may convert the first image feature into a hash code, so as to obtain the first hash code. And the terminal may convert the second image feature into a hash code after acquiring the second image feature from the database, so as to obtain the second hash code corresponding to each second image feature; All the second image features in the database are converted into second hash codes corresponding to the second image features, and then the second hash codes are stored in association with the second image features to generate an image feature library, so that in the generation stage of the image feature library, The hash function set can be trained and learned, and when the terminal needs to obtain the second hash code corresponding to the second image feature, it can be directly obtained from the image feature library.

Optionally, after obtaining the first hash code and the second hash code, the terminal further calculates the Hamming distance between the first hash code and the second hash code. Among them, the calculation formula of Hamming distance is:

Here x and y are both n-bit codes,

Represents an exclusive OR operation.

It should be noted that the Hamming distance represents the number of different bits corresponding to two strings (same length); if the XOR operation is performed on the two strings, and the number of the result is 1, then this number is the Hamming distance.

Optionally, after the terminal determines the Hamming distance between the first hash code and the second hash code, it further detects whether the obtained Hamming distance is less than or equal to a preset threshold. The preset threshold is used to measure the size of the Hamming distance, and can be set according to actual needs, for example, the value range can be 0-10; the smaller the Hamming distance, the first hash code and the second hash code The higher the matching degree between the codes, and the Hamming distance is 0, the two are exactly the same.

Optionally, when the terminal detects that the Hamming distance between the first hash code and the second hash code is less than or equal to a preset threshold, it determines that the second image feature corresponding to the second hash code is the same as the first hash code. Image feature matching; when the terminal detects that the Hamming distance between the first hash code and the second hash code is greater than a preset threshold, it determines that the second image feature corresponding to the second hash code and the first image feature Mismatch.

It should be understood that when there are multiple compliance target images that meet the preset conditions, the second hash codes corresponding to the multiple second image features will be obtained, and at this time, the first hash code and each The Hamming distance between the second hash codes is obtained, and the Hamming distances corresponding to a plurality of different second hash codes are obtained. Then the terminal can use the array size sorting, select the Hamming distance with the smallest value from the plurality of Hamming codes, and then execute step S53 based on the selected Hamming distance, so as to use the Hamming distance to determine the second image feature Whether it matches the first image feature; if so, then determine that the second image feature to which the second hash code corresponding to the selected Hamming distance belongs matches the first image feature; if not, then determine that the selected Hamming distance The second image feature to which the second hash code corresponding to the distance belongs does not match the first image feature.

In this way, by using the Hamming distance to realize the matching verification between the first image feature and the second image feature, the operation efficiency of image feature pairing can be improved, thereby improving the process of inspecting the oil and gas pipeline area. The efficiency of whether there are offending targets in the target image.

In an embodiment, on the basis of the above-mentioned embodiment, after the step of determining that the target image is an abnormal target image, the method further includes:

Step S80, outputting the target image to an associated device;

Step S81, when receiving a positive response fed back by the associated device to the target image, update the target image to the compliance target image, and generate the reference image sample based on the target image; or,

Step S82, when receiving a negative response from the associated device for the target image, maintaining the determination that the target image is the abnormal target image.

In this embodiment, when the terminal determines that the target image is an abnormal target image, in order to avoid a misjudgment of a newly built compliant building that has not been recorded in the library, the target image can be output to the relevant work responsible for supervising and maintaining oil and gas pipelines The associated equipment of the personnel is transferred to the manual review process for the relevant staff to review whether the target in the target image is an abnormal target.

Optionally, when the relevant staff checks that the target in the target image is a compliant target, a positive response for feedback on the target image may be sent to the terminal through the associated device. When the terminal receives a positive response from the associated device for the target image, indicating that the target in the target image should be a compliant target, the terminal can update the previous current target image to a compliant target image, and based on the The target image generates a reference image sample, and the newly generated reference image sample is input into the neural network model to train and update the neural network model, so that the neural network model can add relevant training parameters of the compliant target image. When the same target image is identified, a second image feature matching the first image feature of the target image can be obtained, that is, it will no longer be determined as a non-abnormal target image, and it is determined as a compliant target image.

Optionally, when the relevant staff checks that the target in the target image is an abnormal target (for example, an illegal building), a negative response for feedback on the target image may be sent to the terminal through the associated device. When the terminal receives a negative response from the associated device to the target image, indicating that the target in the target image is indeed an abnormal target, the terminal continues to maintain the previous determination that the target image is an abnormal target image, that is, at this time The terminal can be left alone.

In this way, only when the target image is identified as a non-compliant target image, the process of manually assisting the review of whether the target image has abnormal targets will be transferred, which reduces the cost of manually reviewing whether the targets in the target image are abnormal targets to a certain extent. , which improves the efficiency of identifying illegal targets in the oil and gas pipeline area, and also avoids misjudging the existence of abnormal targets in the target image.

In an embodiment, on the basis of the above-mentioned embodiment, after the step of determining that the target image is an abnormal target image, the method further includes:

Step S90: Generate alarm information according to the abnormal target image, and output the alarm information.

In this embodiment, when the terminal determines that the target image is an abnormal target image, alarm information may be generated according to the abnormal target image. Wherein, in the alarm information, the terminal may further add the first position associated with the abnormal target image, and/or add the aerial image to which the abnormal target image belongs and the position information associated with the aerial image.

Further, the terminal can further determine the information receiving end of relevant law enforcement agencies (such as public security departments, urban management departments, relevant departments responsible for oil and gas pipeline maintenance, etc.), and output the alarm information to the information receiving end to prompt relevant law enforcement agencies to deal with oil and gas in time. Illegal, illegal and illegal buildings near the pipeline can avoid the loss of personnel, economy and environment caused by damage to the oil and gas pipeline as soon as possible.

Referring to FIG. 2, the embodiment of the present application also provides a target matching-based oil and gas pipeline inspection device 10, including:

The acquisition module 11 is used for collecting aerial images in the oil and gas pipeline area by using the drone;

a processing module 12, for extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

The feature extraction module 13 is configured to perform image feature extraction on the target image by using a neural network model to obtain a first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are labeled There is a compliance target image; the neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

A detection module 14, configured to detect a compliance target image that satisfies a preset condition according to the second position and the first position corresponding to the compliance target in the compliance target image;

A judgment module 15, configured to judge whether the detected second image feature corresponding to the compliance target image matches the first image feature;

The first determination module 16 is configured to determine if the target image is the compliance target image;

The second determination module 17 is configured to determine that the target image is an abnormal target image if not.

In an embodiment, on the basis of the above-mentioned embodiment, the judgment module of the oil and gas pipeline inspection device based on target matching includes:

an obtaining unit, configured to obtain the first hash code of the first image feature, and the second hash code of the second image feature corresponding to the detected compliance target image;

a determining unit for determining the Hamming distance between the first hash code and the second hash code;

A judgment unit, configured to use the Hamming distance to judge whether the second image feature matches the first image feature.

Further, the judging unit is further configured to judge whether the Hamming distance is less than or equal to a preset threshold; if so, judge that the second image feature matches the first image feature; if not, judge the The second image feature does not match the first image feature.

Further, the neural network model is constructed based on the Mask RCNN network; the processing module is also used to input the aerial image into the neural network model, to utilize the Mask RCNN network to extract from the aerial image. target image; wherein, the Mask RCNN network is also used to extract the first image feature and the second image feature.

Further, the preset condition is any one of the following: the second position is within a preset area constructed based on the first position; the distance between the second position and the first position is less than or equal to the preset distance.

Further, the oil and gas pipeline inspection device based on target matching also includes:

A third determination module, configured to determine that the target image is an abnormal target image when the compliance target image that meets the preset condition is not detected.

Further, the oil and gas pipeline inspection device based on target matching also includes:

an output module for outputting the target image to an associated device;

a first receiving module, configured to update the target image to the compliance target image when receiving a positive response from the associated device to the target image, and generate the reference image sample based on the target image ;or,

The second receiving module is configured to maintain the determination that the target image is the abnormal target image when receiving a negative response fed back by the associated device to the target image.

Further, the oil and gas pipeline inspection device based on target matching also includes:

An alarm module, configured to generate alarm information according to the abnormal target image, and output the alarm information.

Referring to FIG. 3 , an embodiment of the present application further provides a computer device. The computer device may be a server, and its internal structure may be as shown in FIG. 3 . The computer device includes a processor, memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for target matching based oil and gas pipeline inspection procedures. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for inspecting oil and gas pipelines based on target matching is realized.

Those skilled in the art can understand that the structure shown in FIG. 3 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied.

In addition, the present application also proposes a computer-readable storage medium, the computer-readable storage medium includes an oil and gas pipeline inspection program based on target matching, and the oil and gas pipeline inspection program based on target matching is executed by a processor. The steps of the oil and gas pipeline inspection method based on target matching described in the above embodiments. It can be understood that the computer-readable storage medium can be non-volatile or volatile.

To sum up, the oil and gas pipeline inspection method based on target matching, the oil and gas pipeline inspection device based on target matching, computer equipment and storage medium provided in the embodiments of the application utilize artificial intelligence and target detection technology to automatically Perform image recognition and feature extraction on the images in the oil and gas pipeline area collected by man and machine to obtain the target building in the image, and use the image features corresponding to the compliant buildings that are close to the target building. Image features are used for feature matching, and based on this, it is judged whether the target building is a compliant building, and if not, it is an illegal building, thus improving the efficiency of identifying illegal targets in the oil and gas pipeline area during the process of patrolling the oil and gas pipeline area. .

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium provided in this application and used in the embodiments may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article or method comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, apparatus, article or method. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, apparatus, article, or method that includes the element.

The above are only the preferred embodiments of the present application, and are not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, or directly or indirectly applied to other related The technical field is similarly included in the scope of patent protection of this application.

Claims (20)

1. An oil and gas pipeline inspection method based on target matching, comprising:

   Use drones to collect aerial images in the oil and gas pipeline area;

   extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

   Use a neural network model to perform image feature extraction on the target image to obtain the first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliant target images; The neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

   According to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

   judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

   If so, determine that the target image is the compliance target image;

   If not, it is determined that the target image is an abnormal target image.
2. The oil and gas pipeline inspection method based on target matching according to claim 1, wherein the step of judging whether the second image feature corresponding to the detected compliance target image matches the first image feature comprises:

   obtaining the first hash code of the first image feature, and obtaining the second hash code of the second image feature corresponding to the detected compliance target image;

   determining the Hamming distance between the first hash code and the second hash code;

   Whether the second image feature matches the first image feature is determined by using the Hamming distance.
3. The oil and gas pipeline inspection method based on target matching according to claim 1, wherein, the neural network model is constructed based on the Mask RCNN network; the step of extracting the target image from the aerial image comprises:

   Inputting the aerial image into the neural network model to extract a target image from the aerial image using the Mask RCNN network;

   Wherein, the Mask RCNN network is also used to extract the first image feature and the second image feature.
4. The oil and gas pipeline inspection method based on target matching according to claim 1, wherein the preset condition is any one of the following:

   the second position is within a preset area constructed based on the first position;

   The distance between the second position and the first position is less than or equal to a preset distance.
5. The oil and gas pipeline inspection method based on target matching according to claim 4, wherein after the step of extracting the target image from the aerial image and determining the first position corresponding to the target in the target image, further include:

   When the compliant target image that satisfies the preset condition is not detected, it is determined that the target image is an abnormal target image.
6. The oil and gas pipeline inspection method based on target matching according to claim 1, wherein after the step of determining that the target image is an abnormal target image, the method further comprises:

   outputting the target image to an associated device;

   When receiving a positive response fed back by the associated device to the target image, update the target image to the compliance target image, and generate the reference image sample based on the target image; or,

   When receiving a negative response fed back by the associated device for the target image, the determination that the target image is the abnormal target image is maintained.
7. The oil and gas pipeline inspection method based on target matching according to claim 1, wherein after the step of determining that the target image is an abnormal target image, the method further comprises:

   Generate alarm information according to the abnormal target image, and output the alarm information.
8. An oil and gas pipeline inspection device based on target matching, comprising:

   The acquisition module is used to collect aerial images in the oil and gas pipeline area by using drones;

   a processing module for extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

   The feature extraction module is used for extracting image features of the target image by using a neural network model to obtain a first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with a compliance target image; the neural network model performs image feature extraction on the compliance target image during training to obtain a second image feature;

   a detection module, configured to detect a compliance target image that satisfies a preset condition according to the second position and the first position corresponding to the compliance target in the compliance target image;

   a judgment module for judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

   a first determination module, configured to determine if the target image is the compliance target image;

   The second determination module is configured to determine that the target image is an abnormal target image if not.
9. A computer equipment, wherein, the computer equipment comprises a memory, a processor and a target matching-based oil and gas pipeline inspection program that is stored on the memory and can be run on the processor, and the target-matched oil and gas pipeline inspection program When the pipeline inspection program is executed by the processor, a target matching-based oil and gas pipeline inspection method is realized;

   Wherein, the steps of the oil and gas pipeline inspection method based on target matching include:

   Use drones to collect aerial images in the oil and gas pipeline area;

   extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

   Use a neural network model to perform image feature extraction on the target image to obtain the first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliant target images; The neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

   According to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

   judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

   If so, determine that the target image is the compliance target image;

   If not, it is determined that the target image is an abnormal target image.
10. The computer device according to claim 9, wherein the step of judging whether the second image feature corresponding to the detected compliance target image matches the first image feature comprises:

    obtaining the first hash code of the first image feature, and obtaining the second hash code of the second image feature corresponding to the detected compliance target image;

    determining the Hamming distance between the first hash code and the second hash code;

    Whether the second image feature matches the first image feature is determined by using the Hamming distance.
11. The computer equipment according to claim 9, wherein, the neural network model is constructed based on the Mask RCNN network; the step of extracting the target image from the aerial image comprises:

    Inputting the aerial image into the neural network model to extract a target image from the aerial image using the Mask RCNN network;

    Wherein, the Mask RCNN network is also used to extract the first image feature and the second image feature.
12. The computer device of claim 9, wherein the preset condition is any one of the following:

    the second position is within a preset area constructed based on the first position;

    The distance between the second position and the first position is less than or equal to a preset distance.
13. The computer device according to claim 12, wherein after the step of extracting the target image from the aerial image and determining the first position corresponding to the target in the target image, the method further comprises:

    When the compliant target image that satisfies the preset condition is not detected, it is determined that the target image is an abnormal target image.
14. The computer device according to claim 9, wherein after the step of determining that the target image is an abnormal target image, it further comprises:

    outputting the target image to an associated device;

    When receiving a positive response fed back by the associated device to the target image, update the target image to the compliance target image, and generate the reference image sample based on the target image; or,

    When receiving a negative response fed back by the associated device for the target image, the determination that the target image is the abnormal target image is maintained.
15. A computer-readable storage medium, wherein the computer-readable storage medium stores an oil and gas pipeline inspection program based on target matching, and the oil and gas pipeline inspection program based on target matching realizes target-based matching when the oil and gas pipeline inspection program based on target matching is executed by a processor oil and gas pipeline inspection method;

    Wherein, the steps of the oil and gas pipeline inspection method based on target matching include:

    Use drones to collect aerial images in the oil and gas pipeline area;

    extracting a target image from the aerial image, and determining a first position corresponding to the target in the target image;

    Use a neural network model to perform image feature extraction on the target image to obtain the first image feature, wherein the neural network model is pre-trained based on a plurality of reference image samples, and the reference image samples are marked with compliant target images; The neural network model performs image feature extraction on the compliance target image during the training process to obtain a second image feature;

    According to the second position and the first position corresponding to the compliance target in the compliance target image, detecting a compliance target image that satisfies a preset condition;

    judging whether the detected second image feature corresponding to the compliance target image matches the first image feature;

    If so, determine that the target image is the compliance target image;

    If not, it is determined that the target image is an abnormal target image.
16. The computer-readable storage medium of claim 15, wherein the step of judging whether the second image feature corresponding to the detected compliance target image matches the first image feature comprises:

    obtaining the first hash code of the first image feature, and obtaining the second hash code of the second image feature corresponding to the detected compliance target image;

    determining the Hamming distance between the first hash code and the second hash code;

    Whether the second image feature matches the first image feature is determined by using the Hamming distance.
17. The computer-readable storage medium of claim 15, wherein the neural network model is constructed based on a Mask RCNN network; the step of extracting the target image from the aerial image comprises:

    Inputting the aerial image into the neural network model to extract a target image from the aerial image using the Mask RCNN network;

    Wherein, the Mask RCNN network is also used to extract the first image feature and the second image feature.
18. The computer-readable storage medium of claim 15, wherein the preset condition is any one of the following:

    the second position is within a preset area constructed based on the first position;

    The distance between the second position and the first position is less than or equal to a preset distance.
19. The computer-readable storage medium according to claim 18, wherein after the step of extracting the target image from the aerial image and determining the first position corresponding to the target in the target image, the method further comprises:

    When the compliant target image that satisfies the preset condition is not detected, it is determined that the target image is an abnormal target image.
20. The computer-readable storage medium of claim 15, wherein after the step of determining that the target image is an abnormal target image, the method further comprises:

    outputting the target image to an associated device;

    When receiving a positive response fed back by the associated device to the target image, update the target image to the compliance target image, and generate the reference image sample based on the target image; or,

    When receiving a negative response fed back by the associated device for the target image, the determination that the target image is the abnormal target image is maintained.

Images