CN116721339B

CN116721339B - Method, device, equipment and storage medium for detecting power transmission line

Info

Publication number: CN116721339B
Application number: CN202310452550.6A
Authority: CN
Inventors: 张惠荣; 廖承就; 陈华超; 魏莱; 黄戬; 宋明择; 林杰欢; 陈奕戈; 陈晓儒; 钟振鑫; 邓小康; 邱汉锋; 张建锋; 吴树宁; 李俊杰; 王潇潇; 蓝钢; 王峻; 王瑞麟; 佘仰腾
Original assignee: Guangdong Power Grid Co Ltd; Huizhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Current assignee: Guangdong Power Grid Co Ltd; Huizhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority date: 2023-04-24
Filing date: 2023-04-24
Publication date: 2024-04-30
Anticipated expiration: 2043-04-24
Also published as: CN116721339A

Abstract

The invention discloses a method, a device, equipment and a storage medium for detecting a power transmission line. The method comprises the following steps: acquiring a group of binocular images shot by a binocular camera aiming at a target power transmission line, wherein the binocular images comprise a left-eye image and a right-eye image; determining the actual distance from a target object in the binocular image to a binocular camera according to the left-eye image and the right-eye image aiming at the binocular image to be processed of each frame; if the actual distance is smaller than or equal to the preset distance, taking the binocular image as a first reference image, and determining the stay time of the target object according to the first reference image and the second reference image; if the residence time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level. The method solves the problem that the existing detection method of the power transmission line is low in accuracy of risk detection, improves accuracy of peripheral risk detection of the power transmission line, and provides guarantee for safety of the power transmission line.

Description

Method, device, equipment and storage medium for detecting power transmission line

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting a power transmission line.

Background

External damage is one of the main reasons for accidents of the power transmission line, and can cause great potential safety hazards for safe and stable operation of the power grid. Wherein the external damage includes unreliability of lightning stroke, hurricane, icing, etc. and artificial external damage. The artificial external damage mainly refers to the damage of a power transmission line in the process of constructing the power transmission line by large-scale machines such as a tower crane, a crane, an excavator or a bulldozer.

Compared with the unpredictability of the unreliability, the artificial external force damage of large-scale mechanical illegal construction and the like can be avoided. At present, a target detection model based on a convolutional neural network is used for detecting hidden danger of external force damage of a power transmission line, so that the detection efficiency can be improved while the detection accuracy of the hidden danger target is improved. However, due to the wide field of view of the camera, false detection situations often occur in which the target detection model will detect construction machinery that is far from the transmission line.

Disclosure of Invention

The invention provides a method, a device, equipment and a storage medium for detecting a power transmission line, so as to improve the accuracy of risk detection of the power transmission line.

According to an aspect of the present invention, there is provided a method for detecting a power transmission line, the method including:

acquiring a group of binocular images shot by a binocular camera aiming at a target power transmission line, wherein the binocular images comprise a left-eye image and a right-eye image;

determining the actual distance from a target object in the binocular image to a binocular camera according to the left-eye image and the right-eye image aiming at the binocular image to be processed of each frame;

If the actual distance is smaller than or equal to the preset distance, taking the binocular image as a first reference image, and determining the stay time of the target object according to the first reference image and a second reference image, wherein the second reference image is an image with the acquisition time positioned behind the first reference image, and the actual distance from the target object in the second reference image to the binocular camera is smaller than or equal to the preset distance;

if the stay time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level.

According to another aspect of the present invention, there is provided a detection apparatus for a power transmission line, the apparatus comprising:

The binocular image acquisition module is used for acquiring a group of binocular images shot by the binocular camera aiming at the target power transmission line, wherein the binocular images comprise left-eye images and right-eye images;

The actual distance determining module is used for determining the actual distance from a target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image aiming at each frame of the binocular image to be processed;

the stay time determining module is used for determining the stay time of the target object according to the first reference image and the second reference image, wherein the second reference image is an image with the acquisition time positioned behind the first reference image, and the actual distance from the target object in the second reference image to the binocular camera is smaller than or equal to the preset distance;

The first prompt module is used for generating first prompt information about the target object if the stay time is longer than the preset time, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level.

According to another aspect of the present invention, there is provided an electronic device including:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of detecting a transmission line according to any one of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a method for detecting a power transmission line according to any one of the embodiments of the present invention.

According to the technical scheme provided by the embodiment of the invention, a group of binocular images shot by the binocular camera aiming at the target power transmission line is obtained, and as the binocular images comprise the left-eye image and the right-eye image, the situation of real observation of eyes can be simulated, and more information aiming at the power transmission line is captured. For each frame of binocular image to be processed, determining the actual distance from the target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image, and determining the actual distance from the target object through the binocular image simply, quickly and effectively. If the actual distance is smaller than or equal to the preset distance, taking the binocular image as a first reference image, and determining the stay time of the target object according to the first reference image and the second reference image so as to effectively distinguish objects staying for a long time and objects passing through a short time, and eliminating objects far from the power transmission line and assisting in capturing suspicious objects. If the residence time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level, and can prompt in time when the risk exists, so that the problem that the existing power transmission line detection method is low in risk detection accuracy is solved, the peripheral risk detection accuracy of the power transmission line is improved, and the safety of the power transmission line is guaranteed.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a method for detecting a power transmission line according to an embodiment of the present invention;

Fig. 2 is a flowchart of another method for detecting a power transmission line according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a model-induced collaborative attention mechanism calculation at YOLOv4 according to an embodiment of the present invention;

FIG. 4 is a schematic illustration of an actual distance calculation provided in accordance with an embodiment of the present invention;

Fig. 5 is a flowchart of a specific method for detecting a power transmission line according to an embodiment of the present invention;

fig. 6 is a block diagram of a detection device for a power transmission line according to an embodiment of the present invention;

fig. 7 is a block diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first" and "second" and the like in the description and the claims of the present invention and the above drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a flowchart of a method for detecting a power transmission line according to an embodiment of the present invention, where the embodiment is applicable to a scenario of detecting a risk around a power transmission line based on binocular images, and may be executed by a detection device of the power transmission line, where the detection device of the power transmission line may be implemented in a form of hardware and/or software and configured in a processor of an electronic device.

As shown in fig. 1, the method for detecting the power transmission line includes the following steps:

S110, acquiring a group of binocular images shot by a binocular camera aiming at a target power transmission line, wherein the binocular images comprise a left-eye image and a right-eye image.

The target power transmission line is a power transmission line to be subjected to risk detection, for example, may be a power transmission line with hidden danger of damage caused by external force.

Specifically, in order to detect a large machine near the target transmission line, a binocular camera may be provided near the target transmission line. A binocular camera is used to capture a video for a target transmission line, the binocular image of each frame of the video comprising a left eye image and a right eye image. The left-eye image is an image shot by a left camera of the binocular camera, and the right-eye image is an image shot by a right camera of the binocular camera.

S120, determining the actual distance from a target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image aiming at the binocular image to be processed of each frame.

The target object includes equipment or tools that may damage the target transmission line, and may be, for example, a large-scale machine such as a tower crane, a crane, an excavator, and a bulldozer.

It will be appreciated that the target transmission line may be damaged when large machinery in the vicinity of the target transmission line is under construction. However, the construction range of the large-scale machine is limited, and the large-scale machine is considered to be free from external damage to the target transmission line outside the construction range. Therefore, to exclude large machines distant from the target transmission line, the actual distance of the target object to the binocular camera needs to be determined.

The actual distance from the target object to the binocular camera is determined based on the principle of binocular distance, and the specific embodiment is not particularly limited. The method is characterized in that the angle difference between the two cameras of the binocular camera is determined according to the left eye image and the right eye image, and the actual distance from the target object to the binocular camera is estimated based on the angle difference.

S130, if the actual distance is smaller than or equal to the preset distance, taking the binocular image as a first reference image, and determining the stay time of the target object according to the first reference image and the second reference image.

The second reference image is an image with acquisition time positioned behind the first reference image, and the actual distance from the target object in the second reference image to the binocular camera is smaller than or equal to the preset distance.

Optionally, if the actual distance between the target object and the binocular camera is less than or equal to the preset distance, the target object may cause external damage to the target power transmission line. Further, in order to exclude large machines distant from the target transmission line, a target object whose actual distance is greater than a preset distance may be used as an irrelevant target.

It will be appreciated that the target object may be captured by the binocular camera as it passes around the target transmission line, and therefore, to eliminate false detections caused by this, it is also necessary to determine the target object dwell time. In order to determine the stay time of the target object, an image acquired after the binocular image is acquired, and if the actual distance from the target object to the binocular camera in the image acquired after the binocular image is smaller than or equal to the preset distance, the actual distance is taken as a second reference image. In an embodiment of the present invention, optionally, the residence time of the target object near the target transmission line may be determined according to the number of second reference images and the frame rate of the binocular camera. Or the last second reference image, namely, the last image with the actual distance from the target object to the binocular camera being smaller than or equal to the preset distance is adopted as the target reference image, and the stay time of the target object near the target transmission line is determined according to the corresponding acquisition time of the first reference image and the target reference image.

In a specific embodiment, optionally, segment detection may also be performed. Specifically, the acquired group of binocular images is divided into a plurality of sub-image groups based on a preset time interval or a preset frame number, target detection is performed on the binocular images in each preset time period by taking each sub-image group as a unit, whether the actual distance from a target object to the binocular camera in each preset time period is smaller than or equal to an image with a preset distance, namely a second reference image, is determined, and the stay time of the same target object is further determined according to the acquisition time corresponding to the sub-image group with the second reference image.

It will be appreciated that in a set of consecutive frames of images, it may occur that the actual distance of the target object is first less than or equal to the preset distance, then greater than the preset distance, and finally continues to be less than or equal to the preset distance. For this case, the image of the set frame number may be continuously acquired after the second reference image, and if there are images of the set frame number in which the actual distance of the target object is greater than the preset distance, the dwell time of the target object is determined according to the second reference image and the first reference image. If the image with the actual distance smaller than or equal to the preset distance of the target object exists in the image with the set frame number, taking the image with the actual distance smaller than or equal to the preset distance of the target object as a second reference image, continuing to acquire the image with the set frame number until the actual distance of the target object is larger than the preset distance in the image with the set frame number, taking the last image with the actual distance smaller than or equal to the preset distance of the target object as a second reference image, and determining the stay time of the target object according to the second reference image and the first reference image.

Alternatively, an upper frame number limit may be set, and when the difference in frame number from the first reference image to the second reference image is greater than or equal to the set upper frame number limit, the next image of the second reference image is not acquired any more, at this time, the dwell time of the target object may be determined as a preset value.

The upper limit of the frame number is 90 frames, the second reference image is an image of continuous 90 frames after the first reference image, and the stay time of the target object near the target transmission line is further determined according to the frame numbers of the first reference image and the second reference image and the frame rate of the binocular camera according to the position of the target object in the first reference image and the position of the target object in the image of continuous 90 frames after the first reference image. If the actual distances of the target objects in the 90-frame images are smaller than or equal to the preset distance, the stay time of the target objects is determined to be a preset value, wherein the preset value can be the time length consumed for acquiring the 90-frame images.

Further, determining a dwell time of the target object from the first reference image and the second reference image comprises: by means of a multi-target tracking algorithm, the dwell time of the target object is determined from the first reference image and the second reference image.

Specifically, in a video shot by a section of binocular camera for a target transmission line, a plurality of target objects can be tracked simultaneously using a multi-target tracking (Multiple Object Tracking, MOT) algorithm. Alternatively, the multi-target tracking algorithm may be a Simple Online real-time tracking (AND REALTIME TRACKING, SORT) algorithm, a Simple Online real-time tracking with depth-related metrics (Simple Online and real-TIME TRACKING WITH A DEEP association metric, deepsort) algorithm, a joint detection and embedding (Joint Detection and Embedding, JDE) algorithm, or ByteTrack algorithm.

Specifically, in a scene in which the dwell time of the target object is determined according to the first reference image and the second reference image through deepsort algorithm, one is to establish data association according to the appearance and the position of the target object in the first reference image and the second reference image; the other is to extract the image characteristics of the target object in the first reference image and the second reference image through an algorithm, and then establish data association through image characteristic matching.

Illustratively, the appearance of each image object in the first reference image and the second reference image is determined by deepsort algorithm to determine whether the object is the same target object. Further, for the same target object, the position change/movement track of each target object is determined according to the positions of the target object in the first reference image and the second reference image, so that whether each target object stays or not is judged.

And S140, if the stay time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level.

It will be appreciated that when the residence time of the target object in the vicinity of the target transmission line is greater than the preset time, the large machine may start to be constructed, and for this case, a first prompt message is set, which may be any form of prompt message, to remind the user that the target object is in the vicinity of the transmission line, and that the target transmission line is at the first risk level. The method has the advantages that the risk is convenient to intervene by a user in time, the management efficiency of the power transmission line is improved, and the safety is enhanced.

For example, the residence time of the target object is longer than the preset time, and the user can be prompted that the target power transmission line is at the first risk level through preset prompting information in the forms of sound/light and the like.

Optionally, if the residence time is less than or equal to the preset time, generating second prompt information about the target object, where the second prompt information is used to prompt the user that the target power transmission line is at a second risk level, and the first risk level is higher than the second risk level. Specifically, the residence time of the target object is less than or equal to the preset time, which indicates that the residence time of the target object near the target power transmission line is short, but external damage to the target power transmission line is also possible, and considering that the residence time of the target object and the dangerous degree of the target power transmission line may show a positive correlation in some scenes, a second prompt message different from the first prompt message may be set to prompt the user that the target power transmission line is at a second risk level.

In one embodiment, an alarm device is arranged, and if the stay time of the large machine with the actual distance smaller than or equal to the preset distance is smaller than or equal to the preset time, the alarm device is controlled to send an alarm signal with a first duration; if the residence time is longer than the preset time, controlling the alarm device to send out an alarm signal of a second duration; the second time period is longer than the first time period.

For example, a buzzer is provided, and if the residence time of the large machine is less than or equal to the preset time, the buzzer sounds for 20 seconds; when the residence time is longer than the preset time, the buzzer sounds for 40 seconds.

According to the technical scheme provided by the embodiment of the invention, a group of binocular images shot by the binocular camera aiming at the target power transmission line is obtained, and as the binocular images comprise the left-eye image and the right-eye image, the situation that eyes are really observed can be simulated, and more power transmission line information can be captured. For each frame of binocular image to be processed, determining the actual distance from the target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image, and simply, quickly and effectively determining the actual distance from the target object through the binocular image. If the actual distance is smaller than or equal to the preset distance, taking the binocular image as a first reference image, and determining the stay time of the target object according to the first reference image and the second reference image so as to effectively distinguish the object staying for a long time and the object passing through in a short time, and eliminating the target object far from the power transmission line and assisting in capturing the suspicious object. If the residence time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level, and prompting in time when the risk exists, so that the problem that the existing power transmission line detection method is low in risk detection accuracy is solved, the peripheral risk detection accuracy of the power transmission line is improved, and guarantee is provided for the safety of the power transmission line.

Fig. 2 is a flowchart of another method for detecting a power transmission line according to an embodiment of the present invention, where the present embodiment is applicable to a scenario in which a risk around a power transmission line is detected based on a binocular image, and the method for detecting a power transmission line in the present embodiment belongs to the same inventive concept as the method for detecting a power transmission line in the foregoing embodiment, and further describes a process of determining an actual distance from a target object in a binocular image to a binocular camera according to a left-eye image and a right-eye image based on the foregoing embodiment.

As shown in fig. 2, the method for detecting the power transmission line includes:

s210, acquiring a group of binocular images shot by a binocular camera aiming at a target power transmission line, wherein the binocular images comprise a left-eye image and a right-eye image.

S220, respectively determining a first pixel point and a second pixel point corresponding to the target object in the left-eye image and the right-eye image aiming at the binocular image to be processed of each frame.

The first pixel point and the second pixel point correspond to the same position of the target object.

It will be appreciated that in order to determine the actual distance of the target object to the binocular camera from the binocular image, the actual distance of the target object to the binocular camera in each frame of the binocular image needs to be determined.

Firstly, in a left-eye image, determining all pixel points of a target object, wherein each pixel point is used as a first pixel point; then, in the right-eye image, the pixel points at the same position corresponding to each first pixel point are determined as second pixel points. Optionally, in the right-eye image, determining all pixel points of the target object, and taking each pixel point as a first pixel point; in the left-eye image, the pixel points at the same position corresponding to each first pixel point are determined to be the second pixel points.

The method has the advantages that the points needing to determine the actual distance are limited to the first pixel point and the second pixel point of the target object, the calculation amount of the distance calculated based on the left eye image and the right eye image in the next step can be reduced, the actual distance is calculated only for the pixel points of the target object, and the speed and the accuracy of the peripheral risk detection of the power transmission line are improved.

Further, determining a first pixel point and a second pixel point corresponding to the target object in the left eye image and the right eye image respectively includes:

First, one of the left-eye image and the right-eye image is used as a first image, and the other is used as a second image.

Specifically, in order to determine whether a target object exists in the binocular images, one of the binocular images needs to be selected and target detection is performed on the one of the binocular images. Illustratively, a left eye image is taken as a first image and a right eye image is taken as a second image; the right-eye image may be a first image and the left-eye image may be a second image.

Then, the first image is input into a pre-trained target detection model to determine a target area image corresponding to the target object, wherein the target detection model comprises a YOLOv model.

Optionally, the object detection model is used to determine the type, number and location of the object in the first image, and may be, for example, a fast R-CNN, SSD (Single Shot MultiBox Detector) or single-shot detector (you only look once, YOLO) series model.

In a specific embodiment, the specific structure of the target detection model may include a feature extraction network and a prediction network, where the feature extraction network is used to extract feature data of the first image, input the feature data of the first image into the prediction network, and output the type, number and position of the target object, so as to implement positioning and classification of the target object. Further, an attention mechanism (attention mechanism) may be introduced in the object detection model, based on which information of interest is selected. Alternatively, the attention mechanism may be one of a standard attention mechanism (vanilla attention mechanism), a collaborative attention mechanism (coordinate attention mechanism), a self-attention mechanism (self-attention mechanism), a hierarchical attention mechanism (HIERARCHICAL ATTENTION MECHANISM) or a multi-head attention mechanism (multi-head attention mechanism).

The object detection model is illustratively YOLOv' 4 model, whose structure consists of 4 major parts: a backbone feature extraction network CSPDARKNET network, a spatial pyramid pooling layer SPP layer, a path pooling network PANet network, and a prediction network. The method has the advantages that the CSP and Darknet-53 residual networks are combined, so that the deep network characteristic data of the picture can be extracted, and meanwhile, the calculated amount is reduced; the SPP layer is added to increase receptive field, extract higher-level semantic features, and adapt to target recognition of different sizes and background features. Furthermore, a cooperative attention (coordinate attention, CA) mechanism is added into a CSP module in the CSPDARKNET network, so that the feature expression of the target object can be enhanced, the detection precision of the model on the target object is improved, different weights are given to the transfer channels, the channel with larger weight is dominant in the gradient calculation process, and the important focus part of the network can be detected more obviously in the original image finally.

Fig. 3 is a schematic diagram of a collaborative attention mechanism calculation introduced into a YOLOv model according to an embodiment of the present invention, as shown in fig. 3, first, an improved YOLOv model is input into a left-eye image, and a CA mechanism decomposes an original input feature of the left-eye image into two codes of one-dimensional features, and global pooling is performed on the two aggregated features along two spatial directions of length and width, which has the advantage of establishing a dependency relationship in one spatial direction, and capturing accurate position features in another direction, and the two complementary features are commonly applied to a feature map to enhance the representation of a target object in the left-eye image. And then, respectively encoding the generated feature graphs, superposing the channels, respectively encoding the intermediate feature maps of the space information in the length-width direction through convolution and nonlinear function transformation, splitting the obtained new features according to the values of H and W, and multiplying the obtained new features with the most original input through an activation function to obtain the final attention feature graph.

In a specific embodiment, firstly, training a target detection model by taking images of various types of target objects as samples and taking the types of the target objects and the positions of the target objects in the images as labels to obtain a trained target detection model; inputting the first image into a trained target detection model, and extracting the characteristics of the first image through a characteristic extraction network; further, the target detection model may include a pooling layer for downsampling, which has the advantages of reducing data volume and accelerating operation speed, thereby improving efficiency of peripheral risk detection of the power transmission line. Optionally, a network structure for image scaling may be further set in the target detection model, where the network is specifically configured to scale the first image to a preset size, and predict, by using a prediction network, the target object in the preset size image, to obtain a prediction result about the target object, where the prediction result includes a type, a number, and a position of the target object in the preset size image, so as to implement detection of the target object in the first image. Optionally, a network structure for image segmentation may be further provided, the first image is segmented into a plurality of image blocks with preset sizes, the target objects in the images with preset sizes are predicted, the prediction results of the target objects in the images with preset sizes are obtained, and finally, the prediction results of all the image blocks are combined and output, so that the detection of the target objects is realized. The method has the advantages that when the size of the first image is larger than the preset size, the first image is segmented or scaled to obtain the image with the preset size, and therefore calculation cost and prediction time of the model can be reduced.

For example, first, training an improved YOLOv model by taking various large-scale machine pictures as samples and taking the type and the position of a large-scale machine as a label to obtain a pre-trained model; secondly, inputting a left eye image into a trained model, scaling the left eye image into three channel tensors with preset sizes (such as 416 multiplied by 416) without distortion, and sending the three channel tensors into CSPDARKNET networks to perform primary feature extraction on hidden danger targets; then, the output of the last layer is sent to the SPP layer for carrying out the maximum pooling operation after carrying out convolution for 3 times; and finally, stacking and convolving the information subjected to the maximum pooling, sending the information into PANet, fusing the feature graphs output by the last layer 2, the last layer 3 and the SPP layer in PANet, and finally inputting the fusion result into a prediction network to realize the positioning and classification of the target object. Further, in the prediction network, the prediction network first divides the fusion feature map into s×s grids, each of which is responsible for predicting whether an object whose center falls into the network is a target object, and calculates 3 prediction frames. Each prediction frame corresponds to 5+C values, C represents the number of kinds of suspected target objects, 5 represents information of the prediction frame, and each information is: the coordinates of the center point, the width and height of the circumscribed rectangle frame of the suspected target object and the confidence. And finally, screening by using a non-maximum suppression algorithm, outputting a circumscribed rectangular frame of the suspected target object with high confidence as a final detection result, and taking a region image corresponding to the rectangular frame as a target region image. The advantage of this is that the CA attention mechanism is added into the CSP module in CSPDARKNET and 53 network, which can give different weights to the transfer channels, so that the channel with large weight is dominant in the gradient calculation process, namely the important focused part of target detection in the image.

And finally, registering the first image and the second image based on a preset feature matching model so as to determine a first pixel point and a second pixel point in the first image and the second image according to the target area image.

The feature matching model may be a local feature transformer (Local Feature Transformers, loFTR) model or other model based on Semi-global block matching (Semi-global block matching, SGBM), fast robustness features (Speeded Up Robust Features, SURF) or ORB (Oriented Fast and Rotated Brief) algorithms.

Specifically, firstly, extracting features from a binocular image by adopting a feature matching model; then, registering the binocular image based on the content which is the same or similar to the features of the binocular image, so as to obtain a registered binocular image; and taking all the pixel points of the target area image in the first image as first pixel points, and determining second pixel points corresponding to the first pixel points in the second image according to the relation of pixel point coordinates between registered binocular images. The method has the advantages that the matching relation of the pixel level can be directly learned from the binocular image pair containing the same or similar characteristic content, the extracted characteristic point matching pair has higher precision, and the high-efficiency registration is realized through the parallel computing acceleration of the GPU.

In a specific embodiment, the feature matching model is a LoFTR model. The LoFTR model can generate high-quality feature matching pairs in low-texture and motion-blurred areas, and has a good registration effect on moving target objects. Furthermore, open-source outdoor scene model weights can be imported and directly applied to image registration under the scene of the power transmission line. This has the advantage that LoFTR models can register the first image and the second image without retraining.

For example, first, a binocular image is input to a LoFTR model to generate N feature matching pairs { (a, b) ∈i _li,(a,b)∈I_ri |i=1, 2,3 … N }, where (a, b) ∈i _li is coordinates of feature matching pixels in a left-eye image, (a, b) ∈i _ri is coordinates of feature matching pixels in a right-eye image, and I is a sequence number of the feature matching pairs; and then taking the left-eye image as a first image, matching pixel point coordinates of the characteristics of the target object in the left-eye image according to the coordinate range of the target area image determined by the target detection model in the first image, and acquiring the characteristic matching pixel point coordinates of the target object in the right-eye image according to the sequence number i of the characteristic matching pair, so that the characteristic matching pair of the target area image can be obtained.

It can be understood that feature matching can also be directly performed on the target area image to obtain a feature matching pair of the target area image, so as to obtain a first pixel point of the target object in the target area image, and then determine a second pixel point according to the mapping relation of the binocular image. The method has the advantages that the range of feature matching can be further reduced, the data volume of feature matching is reduced, and the image registration speed is increased.

Further, before registering the first image and the second image based on the preset feature matching model, the method further includes: and carrying out distortion and stereo correction on the first image and the second image based on target parameters of the binocular camera, obtaining a re-projection matrix, and carrying out rotation and re-projection mapping on the re-projection matrix to obtain a first image and a second image to be matched.

It will be appreciated that for the distortion present in the binocular images captured by the binocular camera, correction of the left and right eye images is required before registration is performed.

Firstly, carrying out distortion and three-dimensional correction on a left-eye image and a right-eye image according to internal parameters and external parameters of a binocular camera, and eliminating radial and tangential distortion errors of the binocular image; then, a re-projection matrix is acquired, and a re-projection error is calculated. Specifically, since there is always a distance between the pixel point position calculated according to the projection matrix of the binocular camera and the actual pixel point position, the accumulated sum of the distances is calculated as the reprojection error. And finally obtaining aligned binocular images, namely a first image and a second image to be matched respectively through rotation and reprojection mapping.

In a specific embodiment, the image processing software is used for carrying out distortion and stereo correction on the left-eye image and the right-eye image according to the internal parameters and the external parameters of the binocular camera so as to eliminate radial and tangential distortion errors of the binocular image; obtaining a reprojection matrix, calculating a reprojection error, constructing a cost function based on the reprojection error, optimizing the projection matrix by minimizing the cost function, and finally obtaining an aligned binocular image as a first image and a second image to be matched through rotation and reprojection mapping. The method has the advantages that the re-projection error not only considers the calculation errors of the internal parameters and the external parameters, but also considers the measurement errors of the binocular images, so that the calculation accuracy is improved, and the accuracy of detecting the peripheral risks of the power transmission line is further improved.

Illustratively, using Undistort Points () and StereoRectify () functions in OpenCV, distortion and stereo correction are performed on the left-eye image and the right-eye image according to the internal parameters and the external parameters of the binocular camera to eliminate radial and tangential distortion errors of the binocular image; acquiring a re-projection matrix, calculating a re-projection error, constructing a cost function based on the re-projection error, and optimizing the projection matrix by minimizing the cost function; and rotating and re-projecting the binocular image based on the optimized projection matrix to obtain a first image and a second image to be matched.

S230, determining the actual distance between the target object and the binocular camera according to the first pixel point, the second pixel point and the target parameters of the binocular camera.

The target parameters comprise mapping relation parameters between two cameras of the binocular camera. In a specific embodiment, the mapping relation parameter includes a mapping relation between the first pixel point and a base line (b) and a focal length (f) of the binocular camera and a binocular image acquired by the binocular camera. The pixel value of each first pixel point in the first image and the pixel value of each second pixel point in the second image are respectively determined according to the mapping relation, so that the pixel value difference of each first pixel point and each second pixel point is obtained.

Fig. 4 is a schematic diagram of an actual distance calculation according to an embodiment of the present invention, and two cameras of the binocular camera are horizontally disposed as shown in fig. 4. Firstly, determining a pixel value (XL) of a first pixel point in a horizontal direction according to a preset number of first pixel points of a target object in a first image to be matched; determining a preset number of second pixel points in a second image to be matched according to the mapping relation between the binocular images and the first pixel points, and obtaining a pixel value (XR) of the second pixel points in the horizontal direction; determining the difference between the pixel value (XL) of each point of the target object of the left eye image in the horizontal direction and the pixel value (XR) of the corresponding point of the right eye image in the horizontal direction, and obtaining the actual distance between the target object and the binocular camera according to the formula (1):

Wherein X is the average value of the differences between the preset number XL and the preset number XR, and the actual distance between the target object and the binocular camera is Z.

Further, calibrating the binocular camera to obtain the target parameters includes:

firstly, obtaining distortion coefficients and internal parameters of a binocular camera through a preset monocular calibration algorithm.

Then, image correction is performed on the binocular image based on the internal parameters and the distortion coefficients, and external parameters of the binocular camera are obtained.

Finally, generating target parameters of the binocular camera according to the internal parameters and the external parameters.

The preset monocular calibration algorithm can comprise a calibration algorithm based on a Tsai two-step method or Zhang Zhengyou calibration method and the like, and monocular calibration is carried out on the binocular camera through the preset monocular calibration algorithm to obtain internal parameters and distortion coefficients of the binocular camera. Alternatively, if the binocular image has no distortion, monocular calibration can be performed on the binocular camera by using a method based on active vision, a Kruppa equation or a layered progressive calibration method, so as to obtain internal parameters and external parameters of the binocular camera.

In a specific embodiment, the binocular camera is calibrated based on Zhang Zhengyou calibration methods. Firstly, shooting a plurality of reference images with different angles on a preset reference object by using a binocular camera; monocular calibration is carried out on a left camera and a right camera of the binocular camera respectively through a preset monocular calibration algorithm based on a plurality of reference images, and distortion coefficients and internal parameters of the left camera and the right camera are obtained through operations such as extracting angular points, estimating distortion errors and the like; according to distortion coefficients and internal parameters of the left camera and the right camera, simultaneously according to reference images of the same angle shot by the left camera and the right camera, determining a rotation matrix and a translation matrix between the left camera and the right camera, further determining coordinate points of the same position of a preset reference object in the left camera and the right camera, calibrating the rotation matrix and the translation matrix, and generating an external parameter matrix of the binocular camera according to the calibrated rotation matrix and the calibrated translation matrix; and finally, generating target parameters of the binocular camera according to the internal parameter matrix and the external parameter matrix.

Illustratively, taking a checkerboard as a calibration reference object, and shooting a plurality of checkerboard images with different angles by using a binocular camera; based on a plurality of checkerboard images, performing monocular calibration on a left camera and a right camera of the binocular camera respectively by utilizing camera calibration toolboxes in matlab software, and obtaining distortion coefficients and internal parameter matrixes of the left camera and the right camera through operations such as extracting angular points, estimating distortion errors and the like; taking the distortion coefficients and the internal parameter matrixes of the left camera and the right camera as input, and calling stereoCalibrate functions by utilizing a plurality of corresponding checkerboard images shot by the left camera and the right camera at the same time to output a rotation matrix R and a translation vector T between the left camera and the right camera; removing a binocular image with larger error with the real scale of the checkerboard by using a stereo camera calibrator toolbox, then re-calculating, calibrating a rotation matrix R and a translation vector T, obtaining coordinate points of the same position of the checkerboard in the left-eye image and the right-eye image by means of a checkerboard detection function of OpenCV, and then solving parameter estimation by means of a linear equation to obtain an external parameter matrix of the binocular camera; and finally, generating target parameters of the binocular camera according to the internal parameter matrix and the external parameter matrix.

Furthermore, a preset double-target calibration algorithm may be used to calibrate the double-target camera to obtain target parameters of the double-target camera, which is not specifically limited in this embodiment.

S240, if the actual distance is smaller than or equal to the preset distance, taking the binocular image as a first reference image, and determining the stay time of the target object according to the first reference image and the second reference image.

S250, if the stay time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level.

Fig. 5 is a flowchart of a specific method for detecting a power transmission line according to an embodiment of the present invention, as shown in fig. 5, where the method includes:

s1, acquiring a group of binocular images shot by a binocular camera aiming at a target power transmission line, wherein the binocular images comprise a left-eye image and a right-eye image.

S2, detecting a target object in the left eye image by adopting the improved YOLOv model, and determining a target area image.

The left eye image is input into the improvement YOLOv4 model, the type and the number of the target objects in the left eye image, the rectangular frame including the edges of the target objects and the coordinates of the upper left corner and the lower right corner of the rectangular frame are output, and the image in the rectangular frame including the target objects is taken as the target area image.

And S3, correcting the left-eye image and the right-eye image to obtain aligned binocular images serving as binocular images to be matched.

And S4, performing feature matching on the binocular images to be matched by using the LoFTR model to obtain feature matching pairs of the target area.

Specifically, performing feature matching on the binocular image to be matched by using a LoFTR model to obtain feature matching pairs aiming at the same position in the binocular image; and screening the characteristic matching pairs according to the positions of the coordinates of the upper left corner and the lower right corner of the rectangular frame of the target area image in the binocular image to obtain the characteristic matching pairs of the target area image as the characteristic matching pairs of the target object.

And S5, determining pixel points of the target object on the binocular image based on the feature matching pair, and calculating the actual distance between the target object and the binocular camera.

Specifically, determining a pixel point of the target object on the left-eye image based on the characteristic matching pair of the target object; determining pixel points of a target object on a right-eye image based on a mapping relation between two cameras of the binocular camera, wherein the pixel points in a left-eye image and a right-eye image are pixel points of the same position of the target object; and determining the actual distance between the target object and the binocular camera based on the pixel point of the same position of the target object and the base line and the focal length of the binocular camera.

And S6, carrying out alarm judgment by combining the actual distance and the stay frame number to realize the monitoring of external damage prevention.

When the actual distance is larger than a preset threshold, the target object is regarded as an irrelevant target, when the actual distance is smaller than or equal to the preset distance, the stay frame number of the target object is counted in real time through deepsort algorithm, if the stay frame number of the target object in a power transmission line monitoring area is smaller than or equal to 90 frames, low-risk warning is carried out, and staff is reminded that the potential hazard target enters the scene; if the stay frame number of the target object in the power transmission line monitoring area is greater than 90 frames, high risk warning is carried out, and the trend that the target stays for a long time is indicated.

According to the technical scheme, feature matching is conducted on the target area image, feature matching pixel points of the target object in the target area image are obtained, the actual distance of the target object is determined according to the feature matching pixel points, the problem that the existing detection method of the power transmission line is low in accuracy is solved, the periphery of the power transmission line can be monitored in real time, the feature matching range is reduced, the feature matching data size is reduced, the image registration speed is increased, and the power transmission line detection speed and accuracy are further increased.

Fig. 6 is a block diagram of a power transmission line detection device according to an embodiment of the present invention, where the embodiment is applicable to a scenario of power transmission line peripheral risk detection based on binocular images, and the device may be implemented in hardware and/or software, and integrated into a processor of an electronic device with an application development function.

As shown in fig. 6, the power transmission line detection apparatus includes:

the binocular image acquisition module 501 is configured to acquire a set of binocular images captured by a binocular camera for a target transmission line, where the binocular images include a left-eye image and a right-eye image; the actual distance determining module 502 is configured to determine, for each frame of the binocular image to be processed, an actual distance from a target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image; the stay time determining module 503 is configured to determine, if the actual distance is less than or equal to the preset distance, a stay time of the target object according to the first reference image and the second reference image, where the second reference image is an image whose acquisition time is located after the first reference image, and the actual distance from the target object in the second reference image to the binocular camera is less than or equal to the preset distance; the first prompting module 504 is configured to generate first prompting information about the target object if the residence time is greater than a preset time, where the first prompting information is used to prompt that the target power transmission line is at a first risk level.

Optionally, the device further includes a second prompting module, where the second prompting module is specifically configured to:

If the residence time is smaller than or equal to the preset time, generating second prompt information about the target object, wherein the second prompt information is used for prompting a user that the target power transmission line is at a second risk level, and the first risk level is higher than the second risk level.

Optionally, the actual distance determining module 502 is further configured to:

Respectively determining a first pixel point and a second pixel point corresponding to a target object in a left-eye image and a right-eye image, wherein the first pixel point and the second pixel point correspond to the same position of the target object;

and determining the actual distance between the target object and the binocular camera according to the first pixel point, the second pixel point and the target parameters of the binocular camera, wherein the target parameters comprise the mapping relation parameters between the two cameras of the binocular camera.

Optionally, the actual distance determining module 502 further includes a pixel determining submodule, where the pixel determining submodule is specifically configured to:

taking one of the left eye image or the right eye image as a first image, and the other eye image as a second image;

Inputting the first image into a pre-trained target detection model to determine a target area image corresponding to a target object, wherein the target detection model comprises a YOLOv model;

Registering the first image and the second image based on a preset feature matching model to determine a first pixel point and a second pixel point in the first image and the second image according to the target area image, wherein the feature matching model comprises a LoFTR model, and the first pixel point and the second pixel point correspond to the same position of the target object.

Optionally, the pixel point determining submodule includes an image correction unit, and the image correction unit is specifically configured to: and carrying out distortion and stereo correction on the first image and the second image based on target parameters of the binocular camera, obtaining a re-projection matrix, and carrying out rotation and re-projection mapping on the re-projection matrix to obtain a first image and a second image to be matched.

Optionally, the actual distance determining module 502 further includes a dual-target stator module, which is specifically configured to:

obtaining distortion coefficients and internal parameters of the binocular camera through a preset monocular calibration algorithm;

Performing image correction on the binocular image based on the internal parameters to obtain external parameters of the binocular camera;

And generating target parameters of the binocular camera according to the internal parameters and the external parameters.

Optionally, the residence time determination module 503 is further configured to: the dwell time of the target object is determined from the first reference image and the second reference image by a multi-target tracking algorithm, which includes deepsort algorithm.

The detection device for the power transmission line provided by the embodiment of the invention can execute the detection method for the power transmission line provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Fig. 7 is a block diagram of an electronic device according to an embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 7, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the respective methods and processes described above, such as a transmission line detection method.

In some embodiments, the method of transmission line detection may be implemented as a computer program, which is tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the above-described transmission line detection method may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the transmission line detection method in any other suitable way (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. The method for detecting the power transmission line is characterized by comprising the following steps of:

Determining the actual distance from a target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image aiming at the binocular image to be processed of each frame;

If the actual distance is smaller than or equal to a preset distance, the binocular image is used as a first reference image, the stay time of the target object is determined according to the first reference image and a second reference image, the second reference image is an image with acquisition time positioned behind the first reference image, and the actual distance from the target object in the second reference image to the binocular camera is smaller than or equal to the preset distance;

If the residence time is longer than the preset time, generating first prompt information about the target object, wherein the first prompt information is used for prompting that the target power transmission line is at a first risk level;

Wherein the determining the actual distance from the target object in the binocular image to the binocular camera according to the left eye image and the right eye image comprises:

Respectively determining a first pixel point and a second pixel point corresponding to the target object in the left eye image and the right eye image, wherein the first pixel point and the second pixel point correspond to the same position of the target object;

determining the actual distance between the target object and the binocular camera according to the target parameters of the first pixel point, the second pixel point and the binocular camera, wherein the target parameters comprise the mapping relation parameters between two cameras of the binocular camera;

the determining the first pixel point and the second pixel point corresponding to the target object in the left eye image and the right eye image respectively includes:

Taking one of the left eye image or the right eye image as a first image and the other eye image as a second image;

inputting the first image into a pre-trained target detection model, and determining a target area image corresponding to the target object, wherein the target detection model comprises a YOLOv model;

Registering the first image and the second image based on a preset feature matching model to determine a first pixel point and a second pixel point in the first image and the second image according to the target area image, wherein the feature matching model comprises a LoFTR model.

2. The method of claim 1, further comprising, after said determining a dwell time of said target object from said first and second reference images:

And if the residence time is smaller than or equal to the preset time, generating second prompt information about the target object, wherein the second prompt information is used for prompting a user that the target power transmission line is at a second risk level, and the first risk level is higher than the second risk level.

3. The method of claim 1, further comprising, prior to the registering the first image and the second image based on the preset feature matching model:

And carrying out distortion and stereo correction on the first image and the second image based on the target parameters of the binocular camera to obtain a re-projection matrix, and carrying out rotation and re-projection mapping on the re-projection matrix to obtain a first image and a second image to be matched.

4. The method of claim 1, wherein calibrating the binocular camera to obtain the target parameters comprises:

Performing image correction on the binocular image based on the internal parameters and the distortion coefficients to obtain external parameters of the binocular camera;

5. The method of claim 1, wherein the determining the dwell time of the target object from the first and second reference images comprises:

The dwell time of the target object is determined from the first and second reference images by a multi-target tracking algorithm, including deepsort algorithm.

6. A detection device for a power transmission line, comprising:

The actual distance determining module is used for determining the actual distance from a target object in the binocular image to the binocular camera according to the left-eye image and the right-eye image aiming at the binocular image to be processed of each frame;

The stay time determining module is used for determining the stay time of the target object according to the first reference image and the second reference image, wherein the second reference image is an image with acquisition time positioned behind the first reference image, and the actual distance from the target object in the second reference image to the binocular camera is smaller than or equal to the preset distance;

The first prompting module is used for generating first prompting information about the target object if the residence time is longer than a preset time, wherein the first prompting information is used for prompting that the target power transmission line is at a first risk level;

the actual distance determining module is specifically configured to:

The actual distance determining module further comprises a pixel point determining submodule, wherein the pixel point determining submodule is used for:

7. An electronic device, the electronic device comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of transmission line detection of any one of claims 1-5.

8. A computer readable storage medium, characterized in that the computer readable storage medium stores computer instructions for causing a processor to implement the method for detecting a power transmission line according to any one of claims 1-5 when executed.