CN113345019A - Power transmission line channel hidden danger target ranging method, equipment and medium - Google Patents

Power transmission line channel hidden danger target ranging method, equipment and medium Download PDF

Info

Publication number
CN113345019A
CN113345019A CN202110643955.9A CN202110643955A CN113345019A CN 113345019 A CN113345019 A CN 113345019A CN 202110643955 A CN202110643955 A CN 202110643955A CN 113345019 A CN113345019 A CN 113345019A
Authority
CN
China
Prior art keywords
hidden danger
dimensional
data
target
danger target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110643955.9A
Other languages
Chinese (zh)
Other versions
CN113345019B (en
Inventor
刘伟
蔡富东
吕昌峰
刘焕云
郭国信
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Senter Electronic Co Ltd
Original Assignee
Shandong Senter Electronic Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Senter Electronic Co Ltd filed Critical Shandong Senter Electronic Co Ltd
Priority to CN202110643955.9A priority Critical patent/CN113345019B/en
Publication of CN113345019A publication Critical patent/CN113345019A/en
Application granted granted Critical
Publication of CN113345019B publication Critical patent/CN113345019B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30184—Infrastructure
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Electromagnetism (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the specification discloses a method for measuring distance of a hidden danger target of a power transmission line channel, which comprises the following steps: acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel, and acquiring two-dimensional position information of a hidden danger target according to a target segmentation network model; if the two-dimensional position information of the hidden danger target is in a preset channel three-dimensional protection area, mapping the two-dimensional position information of the hidden danger target to three-dimensional point cloud data through a preset ground data mapping model to obtain three-dimensional coordinate position information of the hidden danger target; and obtaining the distance information between the hidden danger target and the lead according to the three-dimensional coordinate position information of the hidden danger target and a preset lead data mapping model. The method comprehensively utilizes the dense information advantage of the two-dimensional image data and the precision advantage of the three-dimensional point cloud data, and provides more effective early warning information for the safety of the power transmission line channel.

Description

Power transmission line channel hidden danger target ranging method, equipment and medium
Technical Field
The specification relates to the technical field of power transmission lines, in particular to a power transmission line channel hidden danger target ranging method, equipment and medium.
Background
The transmission line channel is extended to hundreds of miles, the environment is complicated, the transmission line channel is influenced by external environment factors all the time, and potential safety hazards appear. Therefore, in operation and maintenance of the power transmission line, monitoring the surrounding environment of the power transmission line is important work, such as monitoring whether ultrahigh trees exist or not, illegal buildings, illegal construction and other environmental hidden dangers affecting the safety of the power transmission line. Meanwhile, with the acceleration of the urbanization process, the construction of the machinery is more and more, and when the shortest distance of the machinery during construction is smaller than the safe distance of the power transmission line, the safety problem of casualties or tripping caused by discharging is likely to occur. In addition, large construction machinery, particularly cranes and cement pump trucks in a lifting arm or extension state, can easily hang up electric wires in the operation process, and thus great threat is brought to the wires. Therefore, in order to guarantee the power transmission safety of the power transmission line channel, it is necessary to identify the hidden dangers and quantitatively and qualitatively determine the threat degree of the lead.
The method mainly adopts an unmanned aerial vehicle shooting and fixed-point monitoring shooting mode to carry out hidden danger detection and danger degree judgment on the power transmission line channel in the prior art. However, the mode of unmanned aerial vehicle inspection shooting is easily influenced by flight factors such as: the influence of radio environment, meteorological environment, geographical environment, and unmanned aerial vehicle patrols and examines the single collection quantity of shooting and is limited, and operating cost is high, gathers the validity and hangs down, is not fit for the transmission line passageway monitoring scene of the faster scene of hidden danger target update. In the current distance measurement technology, a monitoring camera is installed on a power transmission line, regular photographing is performed for inspection, and hidden danger detection is performed by using an image analysis service on an edge device section or a server, so that monitoring center personnel can detect pictures of hidden danger targets and observe state information of the hidden dangers. However, to obtain the information of the hidden danger and the distance between the lead wires, the information needs to be obtained by combining a ranging algorithm to perform post-calculation, and the calculation process is very complex.
Therefore, a distance measuring method for a hidden danger target of a power transmission line, which is low in cost, more convenient and effective, is needed.
Disclosure of Invention
One or more embodiments of the present specification provide a method for measuring a distance of a target with a hidden danger in a channel of a power transmission line, which is used to solve the following technical problems: how to provide a distance measuring method for a hidden danger target of a power transmission line, which has lower cost and is more convenient and effective.
One or more embodiments of the present disclosure adopt the following technical solutions:
one or more embodiments of the present specification provide a method for measuring a distance of a target with a hidden danger in a channel of a power transmission line, where the method includes:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
Optionally, in one or more embodiments of the present specification, before the detecting the hidden danger target on the two-dimensional image data by using a preset target recognition and segmentation network model, the method further includes:
collecting samples of the hidden danger of the power transmission line channel to construct a sample set of the hidden danger of the power transmission line channel;
constructing an initial target recognition segmentation network model, and training the initial target recognition segmentation network model according to the sample set of the hidden danger of the power transmission line channel so as to train a qualified target recognition segmentation network model;
the power transmission line channel hidden danger sample at least comprises one or more of the following items: large-scale construction machinery, towers, tower cranes, trees and buildings.
Optionally, in this description is in one or more embodiments, before determining, according to the position information of the hidden danger target, that the hidden danger target is located in a pre-established channel stereo-protected area, the method further includes:
fitting discrete points of wires in the three-dimensional point cloud data into a three-dimensional curve, and establishing a corresponding wire data mapping model according to the three-dimensional curve; wherein the wire is an edge wire of the transmission line channel;
acquiring three-dimensional coordinate position information of the wire according to the wire data mapping model;
and generating a channel three-dimensional protection area of the transmission line channel according to the three-dimensional coordinate position information of the lead and the voltage grade safety distance of the transmission line channel.
Optionally, in one or more embodiments of the present specification, after determining the position information of the segmented hidden danger target, the method further includes:
acquiring historical two-dimensional image data shot by the monocular camera;
acquiring historical segmentation image data consistent with the position information in the historical two-dimensional image data;
comparing and analyzing the historical segmentation image data and the data information of the hidden danger target;
and if the difference between the data information of the hidden danger target and the historical segmentation image data exceeds a preset rule, updating the three-dimensional point cloud data.
Optionally, in one or more embodiments of the present specification, before the mapping the ground discrete points in the three-dimensional point cloud data through the pre-established ground data mapping model, the method further includes:
calibrating a monocular camera to obtain internal parameters of the monocular camera;
adjusting the visual angle of the three-dimensional point cloud data to enable the three-dimensional point cloud data to be overlapped with the two-dimensional image data acquired by the monocular camera;
the coordinate of the three-dimensional point cloud data is used as a preset reference coordinate, and the three-dimensional point cloud data and the pixel coordinate of the monocular camera are combined through a camera pose estimation algorithm to obtain the external parameters of the three-dimensional point cloud data and the monocular camera;
and obtaining a second space mapping conversion relation between each data in the two-dimensional image data and the three-dimensional point cloud data through the joint calculation of the internal parameters and the external parameters so as to establish a ground data mapping model according to the second space mapping conversion relation.
Optionally, in one or more embodiments of the present specification, the obtaining an actual distance between the hidden danger target and the wire based on the three-dimensional coordinate position information of the hidden danger target and a pre-established wire data mapping model specifically includes:
acquiring three-dimensional coordinate position information of the wire in the three-dimensional point cloud data according to the wire data mapping model;
calculating the distance between the three-dimensional coordinate position information of the highest point of the hidden danger target and the coordinate position information of the lead in the three-dimensional point cloud data to obtain the actual distance between the hidden danger target and the lead
Optionally, in one or more embodiments of the present specification, after obtaining the distance between the hidden danger target and the wire based on the coordinate position information in the three-dimensional point cloud data and a pre-established wire model, the method further includes:
determining the shortest distance in the actual distances according to the actual distances between the hidden danger target and each point in the wire;
outputting the threat level of the hidden danger target according to the shortest distance and the preset clearance distances of different voltage levels of the power transmission line; wherein the threat level is set with a plurality of levels from high to low.
Optionally, in one or more embodiments of the present specification, after determining a shortest distance of the actual distances according to actual distances between the hidden danger target and each point in the wire, the method further includes:
mapping the conducting wires in the three-dimensional point cloud data through a conducting wire mapping conversion relation so as to obtain a third space mapping conversion relation between the three-dimensional coordinate position information of the conducting wires and the two-dimensional image data;
and mapping the wire point corresponding to the shortest distance into two-dimensional image data according to the third space mapping conversion relation, and labeling the wire point in a two-dimensional image.
One or more embodiments of the present specification provide a target ranging apparatus for a hidden danger of a power transmission line channel, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
One or more embodiments of the present specification provide a non-transitory computer storage medium storing computer-executable instructions configured to:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
The embodiment of the specification adopts at least one technical scheme which can achieve the following beneficial effects: the distance measuring method for the hidden danger target and the wire is optimized by utilizing the dense information advantage of two-dimensional image data shot by the monocular camera and the precision advantage of three-dimensional point cloud data. The conversion between the two-dimensional image data and the three-dimensional laser point cloud data is realized through the established second space mapping conversion relation, the distance information between the hidden danger target and the lead can be effectively judged, and the distance measuring function of the hidden danger target is realized. The cost of monitoring the hidden danger target of the power transmission line is reduced, and the effectiveness of hidden danger data is improved. And by judging the danger level of the hidden danger target, more effective early warning information is provided for the safety of the transmission line channel, and the probability of false alarm is greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort. In the drawings:
fig. 1 is a schematic flow chart of a power transmission line channel hidden danger target ranging method provided in an embodiment of the present specification;
fig. 2 is a schematic diagram illustrating a hidden danger target in an application scenario according to an embodiment;
fig. 3 is a schematic flowchart of a monocular camera and laser point cloud joint calibration provided in an embodiment of the present disclosure;
fig. 4(a) is a schematic diagram of a two-dimensional image data view angle in an application scenario according to an embodiment of the present disclosure;
fig. 4(b) is a schematic diagram of a three-dimensional point cloud data view in an application scenario according to an embodiment of the present disclosure;
FIG. 5 is a transformation diagram illustrating a second spatial mapping transformation relationship provided by an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a process for implementing target ranging of hidden danger of a channel of a power transmission line according to an embodiment of the present disclosure;
fig. 7 is an internal structural schematic diagram of a target distance measuring device for hidden danger in a power transmission line channel provided in an embodiment of the present disclosure;
fig. 8 is a schematic diagram of an internal structure of a nonvolatile storage medium according to an embodiment of the present disclosure.
Detailed Description
The embodiment of the specification provides a method, equipment and medium for measuring distance of a hidden danger target of a power transmission line channel.
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present specification without any creative effort shall fall within the protection scope of the present specification.
In operation and maintenance of the power transmission line, monitoring the environment around the power transmission line is an important work. Traditional environmental monitoring generally adopts the mode of artifical the patrolling and examining to go on, and the discernment of transmission line passageway hidden danger distance and dangerous degree adopt unmanned aerial vehicle to shoot and realize through the mode of fixed point supervision more.
Aiming at the existing inspection mode, the traditional manual inspection is implemented manually. Due to the problems of large workload, wide range, high difficulty and the like of manual inspection, the phenomenon of insufficient inspection or substandard inspection is easily caused. And the period of manual inspection is long, so that the problem of supervision blank exists. And the mode that unmanned aerial vehicle patrolled and examined the shooting receives for example: the flight environment factors such as radio environment, meteorological environment and geographic environment are too many uncontrollable factors. Secondly, the unmanned aerial vehicle patrols and examines that the data bulk of mode single collection is limited, the validity of collection is low, with high costs. And the quality of the picture needs to be checked after the picture is collected, when the condition of overexposure or underexposure exists, the complementary shooting needs to be carried out, the three-dimensional reconstruction in the later period needs to be carried out, the data measurement is carried out by using 3D software, and the distance measurement process and the steps are very complicated.
At present, methods for acquiring distance information in a power transmission line scene mainly include passive distance measurement sensing and active distance measurement sensing. The passive distance measurement sensing is mainly based on a visual mode, one mode is a monocular distance measurement algorithm, the algorithm relies on a specific scale to calculate the depth information of a target, the precision is poor, the passive distance measurement sensing is easily interfered by the outside, and the robustness is poor; the other is a multi-view ranging algorithm, which is represented by a binocular ranging algorithm, although the algorithm is independent of a specific scale, the algorithm is limited by hardware, the ranging range is generally within 20m, the accuracy is influenced by hardware equipment, and although a disparity map obtained through a stereo matching algorithm can obtain approximate three-dimensional information of a scene, the disparity of partial pixels has a large error.
In order to solve the above problems, the present specification provides a method for measuring a distance of a target with a hidden danger in a channel of a power transmission line. According to the method, for the camera equipment installed on the monitoring terminal, the data error in the range finding process of the hidden danger target is reduced and the effectiveness of the hidden danger target data is improved through the dense information advantage and the laser point cloud precision advantage of the two-dimensional image data. And obtaining a second space mapping conversion relation between the three-dimensional point cloud data and the two-dimensional image data by jointly calibrating the three-dimensional point cloud data and the two-dimensional image data. And establishing a three-dimensional protection area of the power transmission line channel through the second space mapping conversion relation, and updating the three-dimensional point cloud data according to the type of the hidden danger targets and the distribution of the hidden danger targets, thereby ensuring the effectiveness of the three-dimensional point cloud data. In addition, the hidden danger target is positioned in the two-dimensional image data, so that the dependence on the three-dimensional point cloud can be greatly reduced, the measurement precision is ensured, and the cost in the measurement process is effectively reduced. And the distance information between the hidden danger target and the lead is judged in the three-dimensional space, so that the process of post-calculation processing is omitted. The method improves the efficiency and effectiveness of detection and distance measurement of the hidden danger targets of the power transmission line channel, provides safe and effective early warning information for the power transmission line channel, and reduces the detection cost and human resources.
One or more embodiments of the present description may perform the following steps by a server built in a power line channel hidden danger target ranging system.
The technical solution provided by the present specification is described below with reference to the accompanying drawings.
As shown in fig. 1, fig. 1 is a schematic flowchart of a method for measuring a distance of a target with hidden danger in a power transmission line channel according to one or more embodiments of the present disclosure, where the steps in the method may be executed by a corresponding distance measurement server.
The process in fig. 1 may include the following steps:
s101: acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image data is captured by a monocular camera.
The server acquires two-dimensional image data of the power transmission line channel shot by the monocular camera based on wired transmission and/or wireless transmission, and acquires three-dimensional point cloud data acquired by the three-dimensional laser scanner. And carrying out subsequent hidden danger target ranging on the power transmission line channel according to the data of the hidden danger target contained in the two-dimensional image data and the three-dimensional point cloud data.
Among them, it should be noted that: point cloud data (point cloud data) refers to a collection of vectors in a three-dimensional coordinate system. The scan data is recorded in the form of dots, each dot containing three-dimensional coordinates, some of which may contain color information or reflection intensity information. And a point cloud data set-a point cloud is a three-dimensional data that may be referred to as a three-dimensional point cloud data. And the three-dimensional point cloud data has good environmental robustness, and the acquired data has high precision. However, the three-dimensional point cloud data acquired by the laser scanner is often too discretized for the environment, and the acquired data has large granularity and limited information amount, so that the three-dimensional point cloud data is difficult to be used for applications such as environment understanding. The three-dimensional point cloud data is high in acquisition cost, and the information density contained in the two-dimensional image data is high, so that the acquisition cost is relatively reduced, the two-dimensional image data and the three-dimensional point cloud data are subjected to combined analysis, dependence on the three-dimensional point cloud data can be reduced, the effectiveness and the accuracy of the data in the monitoring process of the power transmission line are improved, and meanwhile, the cost in the monitoring process can be reduced.
S102: detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target.
In one or more embodiments of the present specification, before the detecting a hidden danger target on the two-dimensional image data by using a preset target recognition and segmentation network model, the method further includes:
collecting samples of the hidden danger of the power transmission line channel to construct a sample set of the hidden danger of the power transmission line channel;
constructing an initial target recognition segmentation network model, and training the initial target recognition segmentation network model according to the sample set of the hidden danger of the power transmission line channel so as to train a qualified target recognition segmentation network model;
the power transmission line channel hidden danger sample at least comprises one or more of the following items: large-scale construction machinery, towers, tower cranes, trees and buildings.
And collecting hidden danger samples of the power transmission line channel through historical data stored in the Internet or a memory thereof. After a sufficient number of samples capable of ensuring the accuracy of the model training result are collected, a Mask-RCNN or other example segmentation algorithm models can be adopted to construct the target recognition network model. And training the target recognition network model according to the collected samples to obtain a qualified target recognition segmentation network model.
Among them, it should be noted that: due to the development of modern industrial parks and the acceleration of urbanization processes, large machines such as cranes, cement pump trucks, excavators and the like can have the phenomenon of hanging up and breaking electric wires in the process of lifting arms or excavating, so that electric power facilities such as electric transmission lines and the like are damaged. Secondly, because of the discharge property of the power transmission line, in the building illegally built in the range smaller than the safety range of the power transmission line, the power transmission line and the personal safety are affected by the overhigh trees. Therefore, the potential samples of the power transmission line channel should at least include one or more of the following: large-scale construction machinery, towers, tower cranes, trees and buildings.
It should also be noted that the Mask-RCNN, which is obtained by fast R-CNN extension, extends the classification and regression tasks. And each segmentation task aiming at an ROI (region of interest, ROI for short) is added in the learning model so as to decouple the segmentation task and the classification task and output the position information of the hidden danger target, the segmentation contour of the hidden danger target and the target type of the hidden danger target. Compared with other example segmentation algorithms which perform segmentation and then classify, the method can obtain the segmentation result and the classification result simply and efficiently. The type of the hidden danger target is shown in fig. 2. In one or more embodiments of the present description, a hidden danger target type in a power transmission line scene may be a suspension type in a hidden danger target 1 in fig. 2: crane, etc., or ground type in hidden danger target 2, hidden danger target 3: forklifts, excavators, and the like.
S103: and if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target.
In one or more embodiments of the present specification, before determining, according to the position information of the hidden danger target, that the hidden danger target is located in a pre-established channel stereo-protected area, the method further includes:
fitting discrete points of wires in the three-dimensional point cloud data into a three-dimensional curve, and establishing a corresponding wire data mapping model according to the three-dimensional curve; wherein the wire is an edge wire of the transmission line channel;
acquiring three-dimensional coordinate position information of the wire according to the wire data mapping model;
and generating a channel three-dimensional protection area of the transmission line channel according to the three-dimensional coordinate position information of the lead and the voltage grade safety distance of the transmission line channel.
In one or more embodiments of the present specification, after determining the position information of the segmented hidden danger target, the method further includes:
acquiring historical two-dimensional image data shot by the monocular camera;
acquiring historical segmentation image data consistent with the position information in the historical two-dimensional image data;
comparing and analyzing the historical segmentation image data and the data information of the hidden danger target;
and if the difference between the data information of the hidden danger target and the historical segmentation image data exceeds a preset rule, updating the three-dimensional point cloud data.
In one or more embodiments of the present disclosure, a moving least squares method may be used to fit discrete points of an edge wire to a three-dimensional curve, and a corresponding wire data mapping model may be established according to the three-dimensional curve of the edge wire. And obtaining and fitting three-dimensional coordinate position information of the edge wire according to the wire data mapping model. Due to the high voltage characteristic of the overhead transmission line, a safe distance is kept between the conducting wire and other articles of the building so as to ensure the normal work of the transmission line and the safety of the transmission line. For example: when the 220kV overhead transmission line crosses a building, the minimum vertical distance between a lead and the building is not less than 6 meters; when the 220kV overhead transmission line is close to a building, the minimum clearance between the conducting wire and any point of the building is not less than 5 meters. Therefore, a three-dimensional protection area of the power transmission line channel needs to be generated according to the three-dimensional coordinate position information of the edge wire and by combining safety distances specified by different voltage classes, so that the safety range of the power transmission line channel is determined.
It should be noted that, in the process of fitting the discrete points of the edge wire into a three-dimensional curve, interpolation methods such as curve fitting based on rbf (radial Basis function) and cubic spline curve fitting may also be selected for fitting.
If the position information of the hidden danger target is obtained according to the two-dimensional image data, the hidden danger target can be determined to be in a pre-established channel three-dimensional protection area, and the hidden danger target has safety hazard to a lead in the power transmission line. At this time, the hidden danger target needs to be segmented based on the segmentation contour of the hidden danger target obtained by target segmentation in S102, so as to determine the two-dimensional position information of the segmented hidden danger target and the target type of the hidden danger target.
Because a power transmission line channel has a plurality of movable and unstable target hidden dangers such as a forklift and the like, the three-dimensional laser point cloud data needs to be updated in a fixed period. Compared with two-dimensional image data, the cost for acquiring the three-dimensional point cloud is high. Therefore, in the specification, the hidden danger target is positioned and segmented in the two-dimensional image data, so that the dependence on the three-dimensional point cloud can be greatly reduced, the precision is ensured, and the detection cost can be effectively reduced. The specific implementation manner of the three-dimensional point cloud data updating can be as follows:
and calling and extracting historical two-dimensional image data corresponding to the two-dimensional image data through the monocular camera, and if the historical two-dimensional image data is analyzed and compared with the data information of the hidden danger target, and the difference between the historical two-dimensional image data and the data information of the hidden danger target exceeds a preset rule, updating the current three-dimensional point cloud data to ensure the accuracy and the effectiveness of the detection result. Wherein the preset rule comprises: (1) and if the difference between the occupation ratio of the hidden danger target distribution in the historical two-dimensional data and the occupation ratio of the hidden danger target distribution contained in the current hidden danger target data exceeds 5%, the three-dimensional point cloud data needs to be updated. (2) Because large-scale construction class machinery has the mobility, to the unsettled target that has ground sheltering like hidden danger target 1 shown in fig. 2, there is the hidden danger that the sheltering from that receives the house probably becomes unsettled type. The raising process of the three-dimensional point cloud data has the possibility of hanging up edge wires, the harmfulness is strong, and therefore after the type of the hidden danger target is changed into a suspension type shown as the hidden danger target 1, the data information of the current hidden danger target and the type of the hidden danger target in the historical segmentation image data are changed, the three-dimensional point cloud data need to be updated again, and the safety of the power transmission line is guaranteed. (3) The height and the volume of the building are changed in the illegal building construction process, and when the distribution of the hidden danger targets and the proportion of the hidden danger targets in the image are greatly changed, the point cloud data need to be updated. If the difference between the data of the hidden danger target and the historical two-dimensional image data does not exceed the preset rule, namely the difference can be ignored, and the effectiveness of subsequent analysis cannot be influenced. The three-dimensional point cloud data does not need to be updated so as to save the cost in the point cloud data acquisition process.
S104: mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; and converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation.
In one or more embodiments of the present specification, before the mapping the ground discrete points in the three-dimensional point cloud data through the pre-established ground data mapping model, the method further includes:
calibrating a monocular camera to obtain internal parameters of the monocular camera;
adjusting the visual angle of the three-dimensional point cloud data to enable the three-dimensional point cloud data to be overlapped with the two-dimensional image data acquired by the monocular camera;
the coordinate of the three-dimensional point cloud data is used as a preset reference coordinate, and the three-dimensional point cloud data and the pixel coordinate of the monocular camera are combined through a camera pose estimation algorithm to obtain the external parameters of the three-dimensional point cloud data and the monocular camera;
and obtaining a second space mapping conversion relation between each data in the two-dimensional image data and the three-dimensional point cloud data through the joint calculation of the internal parameters and the external parameters so as to establish a ground data mapping model according to the second space mapping conversion relation.
The data obtained by the three-dimensional point cloud has larger dispersion and larger granularity. The description obtains the intrinsic parameters of the camera through monocular camera calibration to obtain the relationship between the two-dimensional plane pixel coordinate and the three-dimensional world coordinate, so that the information consistency of the two-dimensional image data is utilized to reconstruct three dimensions. On the advantage of guaranteeing the precision of the three-dimensional point cloud data, the characteristic of large discreteness of the three-dimensional point cloud data is compensated, so that the data for carrying out the distance measurement of the hidden danger target is complete, effective and high in precision. The joint calibration process of the two-dimensional image data and the three-dimensional point cloud data is shown in fig. 3. As can be seen from fig. 3, the monocular camera needs to be calibrated to obtain two-dimensional image data of the power transmission channel, and the two-dimensional image data and the three-dimensional point cloud data of the power transmission channel obtained by the three-dimensional point cloud are jointly calibrated, so as to realize modeling of the ground data mapping model and modeling of the conductor data mapping model on the basis of the joint calibration. After the ground data mapping model is constructed, the mapping conversion relation from the two-dimensional image data of the ground data to the three-dimensional point cloud data can be obtained. Meanwhile, according to the wire data mapping model, a channel three-dimensional protection area can be constructed.
First, when calibrating a monocular camera, there are various calibration methods, for example: the monocular calibration can be realized by a Zhang Zhengyou calibration method, an opencv method and the like. In one or more embodiments of the present application, a zhangnyou calibration method is selected to calibrate the camera, and internal parameters of the camera are obtained through a zhangnyou calibration algorithm: f. ofx、fyAs camera focal length parameter, cx、cyIs the camera optical center parameter. Then read threeAnd adjusting the visual angle of the three-dimensional point cloud data by taking the visual angle in the two-dimensional image data as a reference. And selecting clear and obvious points in the two-dimensional image data as characteristic points, and finding the positions of the corresponding characteristic points in the three-dimensional point cloud data. Fig. 4(a) is two-dimensional image data in an application scenario of an embodiment, and fig. 4(b) is a three-dimensional point cloud data image in the application scenario of the embodiment, where the two are the same power transmission line scenario at a uniform viewing angle. As can be seen from fig. 4(a) and 4(b), the feature points selected in this scene are points 1, 2, 3, and 4, and the feature points in the two-dimensional image data correspond to the points in the three-dimensional point cloud data one to one and are the same points.
Among them, it should be noted that: as long as it is determined that the point in the two-dimensional image data and the point in the three-dimensional point cloud data are the same, the point can be used as the feature point, and the position with the large turning degree is preferentially selected as the feature point. And in order to ensure the accuracy of the ranging of the hidden danger target, the point correspondence of the selected characteristic points is more than or equal to 4 groups. And after the visual angle is adjusted, inputting the coordinates of the three-dimensional point cloud serving as preset reference coordinates. In one or more embodiments of the present disclosure, the world coordinate is selected as a preset reference coordinate. After the coordinates of the three-dimensional point cloud are input as world coordinates, the coordinates of the three-dimensional point cloud and the pixel coordinates of the monocular camera can be combined through a camera pose estimation algorithm to obtain external parameters of the three-dimensional point cloud and the monocular camera: rotation matrix R, translation matrix t. And performing combined calculation according to the internal parameters and the external parameters obtained after calibration by the Zhang-Yong algorithm to obtain a second space mapping conversion relation between the three-dimensional point cloud data and the two-dimensional image data so as to obtain a ground data mapping model and a wire data mapping model according to the second space mapping conversion relation, and a first space mapping conversion relation and a third space mapping conversion relation which correspond to the two models respectively.
As shown in fig. 4, four pairs of non-coplanar 3D and 2D feature point pairs are provided, and based on the 4 pairs of feature point pairs, a world coordinate system and a camera coordinate system can be converted through the zhangzhen friend joint calibration and the camera pose estimation described in fig. 5 to obtain a second space mapping conversion relationship between the three-dimensional point cloud data and the two-dimensional image data. The conversion process of the second spatial mapping conversion relationship illustrated in fig. 5 is as follows:
the origin coordinate system of the three-dimensional point cloud is taken as a unified world coordinate system and is defined as Xw，Yw，ZwThe unit is a length unit. The camera coordinate system uses the optical center as the origin of the camera coordinate system and uses the X-direction and the y-direction parallel to the two-dimensional image data as the XCAxis and YCA shaft. And Z isCAxis parallel to optical axis, XC、YC、ZCPerpendicular to each other, the units are length units. The image physical coordinate system takes the intersection point of the main optical axis and the image plane as the coordinate origin, and the x direction and the y direction are shown as the figure, and the unit is a length unit. The image pixel coordinate system takes the vertex of the image as a coordinate origin, and the u direction and the v direction are parallel to the x direction and the y direction, and the unit is in pixel.
World coordinate system (X)w，Yw，Zw) The external parameters determined in the above process: rotating the matrix R and translating the matrix t to perform affine transformation
Figure BDA0003108272150000151
Conversion to camera coordinate System (X)C、YC、ZC). After the transformation from the world coordinate system to the camera coordinate system, the three-dimensional image is transformed to the two-dimensional image to the image coordinate system (x, y) from the perspective projection relationship of 3D-2D determined by the camera pose algorithm in the camera coordinate system. The part with non-coincident coordinates is subjected to proper translation transformation through the adjustment of the internal parameters, and after the coordinate is adjusted to be consistent, the image coordinate system (x, y) is converted into the pixel coordinate system (u, v). And completing the conversion from the world coordinate system to the image coordinate system. Because the three-dimensional point cloud data is used as a preset world coordinate system, the joint calibration between the two-dimensional image data and the three-dimensional point cloud data is realized, and the three-dimensional image data (X) is obtainedw，Yw，Zw) And a second spatial mapping transformation relation with the two-dimensional image data (u, v).
Specifically, the second spatial mapping transformation relationship is obtained by the following calculation from the external parameters and the internal parameters:
Figure BDA0003108272150000161
wherein, R is a rotation matrix, t is a translation matrix, and the two matrixes form a 3 × 4 matrix which is an external parameter matrix of the camera. f. ofx，fy，cx，cyIs an internal parameter of the camera.
In the same manner as the generation of the wire data mapping model described in S103, the discrete points on the ground in the three-dimensional point cloud data are fitted to the three-dimensional curved surface in advance. The second space mapping conversion relation can obtain the conversion relation between each data in the two-dimensional image data and the three-dimensional point cloud data. Therefore, according to the three-dimensional curved surface, a corresponding ground data mapping model can be established, so that the ground data in the three-dimensional point cloud data is mapped according to the ground data mapping model, and a first space mapping conversion relation between the ground data and the two-dimensional image data is obtained. And converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation, and laying a foundation for determining the distance between the hidden danger target and the lead in the three-dimensional space subsequently. The coordinate position information of each data in the three-dimensional point cloud data can be simply and conveniently obtained by carrying out space mapping conversion on the three-dimensional point cloud data and the two-dimensional image data. The complicated steps that after the traditional unmanned aerial vehicle distance measurement process needs to carry out later-stage three-dimensional reconstruction on the underexposed pictures, software is recycled to carry out data analysis and measurement are avoided.
S105: acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
In one or more embodiments of the present description, the obtaining an actual distance between the hidden danger target and the wire based on the three-dimensional coordinate position information of the hidden danger target and a pre-established wire data mapping model specifically includes:
acquiring three-dimensional coordinate position information of the wire in the three-dimensional point cloud data according to the wire data mapping model;
calculating the distance between the three-dimensional coordinate position information of the highest point of the hidden danger target and the coordinate position information of the lead in the three-dimensional point cloud data to obtain the actual distance between the hidden danger target and the lead
In one or more embodiments of the present specification, after obtaining the distance between the hidden danger target and the wire based on the coordinate position information in the three-dimensional point cloud data and a pre-established wire model, the method further includes:
obtaining the shortest distance in the actual distances according to the actual distances between the hidden danger target and each point in the lead;
outputting the threat level of the hidden danger target according to the shortest distance and the preset clearance distances of different voltage levels of the power transmission line; wherein the threat level is set with a plurality of levels from high to low.
In one or more embodiments of the present specification, if it is determined that the hidden danger target is outside a pre-established channel stereo-protected area according to the position information of the hidden danger target, the method further includes: and outputting the threat level of the lowest level according to the hidden danger target.
In one or more embodiments of the present specification, after obtaining a shortest distance in the actual distances according to the actual distances between the hidden danger target and each point in the wire, the method further includes:
mapping the conducting wires in the three-dimensional point cloud data through a conducting wire data mapping model to obtain a third space mapping conversion relation between the three-dimensional coordinate position information of the conducting wires and the two-dimensional image data;
and mapping the wire point corresponding to the shortest distance into two-dimensional image data according to the third space mapping conversion relation, and labeling the wire point in a two-dimensional image.
And acquiring three-dimensional coordinate position information of the lead according to the pre-established lead data mapping model recorded in the S103, and calculating the distance between the three-dimensional position information of the highest point of the hidden danger target and the lead to obtain the distance between the hidden danger target and the lead in the power transmission line. And (4) jointly judging the shortest distance and the preset clearance distances of different voltage levels to output the threat level of the hidden danger target. It should be noted that, according to the voltage levels of different transmission lines, the clearance distances of the wires are different, so that the threat levels can be divided into: urgent, severe, general. Taking a 110KV power transmission line as an example, if the shortest distance between a hidden danger target and a lead is more than 10 meters, the grade is normal; if the shortest distance between the hidden danger target and the lead is 6-10 meters, the threat level is a serious level; and if the shortest distance between the hidden danger target and the lead is less than 6 meters, outputting the threat level as emergency. And secondly, if the hidden danger target is determined to be outside a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, directly outputting the threat level of the hidden danger target to be general.
Meanwhile, a preset wire data mapping model maps the fitted three-dimensional wire in the three-dimensional point cloud data to obtain a third space mapping conversion relation between the three-dimensional coordinate position information of the wire and the two-dimensional image data. And mapping the wire point corresponding to the shortest distance between the hidden danger target and the wire to the two-dimensional image coordinate according to the third space mapping conversion relation so as to label and warn the wire point in the two-dimensional image. In one or more embodiments of the present description, effective early warning information is provided for the safety of a power transmission line channel by further judging the danger level of a hidden danger target, and the validity of a monitoring result is improved by combining three-dimensional point cloud data and two-dimensional image data, so that the probability of false alarm is greatly reduced.
Fig. 6 is a flowchart of a method for measuring a distance to a target with a hidden danger in a power transmission line channel, provided in one or more embodiments of the present specification. As can be seen from fig. 6, in one or more embodiments, when detecting a hidden danger target, the specification needs to acquire two-dimensional image data in a power transmission line channel scene. And detecting the position information of the hidden danger target according to the two-dimensional image data, and if the hidden danger target is determined to be in a preset three-dimensional channel protection area, segmenting the hidden danger target to obtain the contour segmentation data of the hidden danger target. And judging whether the three-dimensional point cloud data needs to be updated or not according to the contour segmentation data of the hidden danger target and the distribution position of the hidden danger points. And if the updating is needed, starting to obtain the updated three-dimensional point cloud data, and updating the ground data mapping model to obtain a second space mapping conversion relation from the two-dimensional image data to the three-dimensional point cloud data. And calculating the distance between the hidden danger target and the lead in the three-dimensional space, and obtaining the threat level of the hidden danger target according to the distance. And if the hidden danger target is determined not to be in the preset three-dimensional channel protection area after the position information of the hidden danger target is detected according to the two-dimensional image data, directly outputting the threat level of the hidden danger target to be general.
Fig. 7 is a schematic diagram of an internal structure of a device for measuring a distance of a target with a hidden danger in a power transmission line channel, provided in one or more embodiments of the present specification.
As can be seen from fig. 7, the apparatus comprises:
at least one processor 701; and the number of the first and second groups,
a memory 702 communicatively coupled to the at least one processor 701; wherein,
the memory 702 stores instructions executable by the at least one processor 701 to enable the at least one processor 701 to:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
Fig. 8 shows a non-volatile storage medium provided in one or more embodiments of the present specification.
As shown in fig. 8, a non-volatile storage medium stores computer-executable instructions 801, the executable instructions 801 comprising:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device, non-volatile computer storage medium embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and in relation to the description, reference may be made to some portions of the description of the method embodiment.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The above description is merely one or more embodiments of the present disclosure and is not intended to limit the present disclosure. Various modifications and alterations to one or more embodiments of the present description will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of one or more embodiments of the present specification should be included in the scope of the claims of the present specification.

Claims (10)

1. A power transmission line channel hidden danger target ranging method is characterized by comprising the following steps:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
2. The method for measuring the distance between the hidden danger targets in the power transmission line channel according to claim 1, wherein before the detection of the hidden danger targets is performed on the two-dimensional image data through a preset target recognition segmentation network model, the method further comprises:
collecting samples of the hidden danger of the power transmission line channel to construct a sample set of the hidden danger of the power transmission line channel;
constructing an initial target recognition segmentation network model, and training the initial target recognition segmentation network model according to the sample set of the hidden danger of the power transmission line channel so as to train a qualified target recognition segmentation network model;
the power transmission line channel hidden danger sample at least comprises one or more of the following items: large-scale construction machinery, towers, tower cranes, trees and buildings.
3. The method for measuring the distance of the hidden danger target of the power transmission line channel according to claim 1, wherein before the hidden danger target is determined to be in a pre-established channel stereo protection area according to the position information of the hidden danger target, the method further comprises the following steps:
fitting discrete points of wires in the three-dimensional point cloud data into a three-dimensional curve, and establishing a corresponding wire data mapping model according to the three-dimensional curve; wherein the wire is an edge wire of the transmission line channel;
acquiring three-dimensional coordinate position information of the wire according to the wire data mapping model;
and generating a channel three-dimensional protection area of the power transmission line channel according to the three-dimensional coordinate position information of the lead and the safety distance corresponding to the corresponding voltage level of the power transmission line channel.
4. The method for measuring the distance of the hidden danger target of the power transmission line channel according to claim 1, wherein after the position information of the segmented hidden danger target is determined, the method further comprises the following steps:
acquiring historical two-dimensional image data shot by the monocular camera;
acquiring historical segmentation image data consistent with the position information in the historical two-dimensional image data;
comparing and analyzing the historical segmentation image data and the data information of the hidden danger target;
and if the difference between the data information of the hidden danger target and the historical segmentation image data exceeds a preset rule, updating the three-dimensional point cloud data.
5. The method for measuring the distance of the target with the hidden danger of the electric transmission line channel is characterized in that before the ground discrete points in the three-dimensional point cloud data are mapped through a pre-established ground data mapping model, the method further comprises the following steps:
calibrating a monocular camera to obtain internal parameters of the monocular camera;
adjusting the visual angle of the three-dimensional point cloud data to enable the three-dimensional point cloud data to be overlapped with the two-dimensional image data acquired by the monocular camera;
the coordinate of the three-dimensional point cloud data is used as a preset reference coordinate, and the three-dimensional point cloud data and the pixel coordinate of the monocular camera are combined through a camera pose estimation algorithm to obtain the external parameters of the three-dimensional point cloud data and the monocular camera;
and obtaining a second space mapping conversion relation between each data in the two-dimensional image data and the three-dimensional point cloud data through the joint calculation of the internal parameters and the external parameters so as to establish a ground data mapping model according to the second space mapping conversion relation.
6. The method according to claim 1, wherein the obtaining of the actual distance between the hidden danger target and the wire based on the three-dimensional coordinate position information of the hidden danger target and a pre-established wire data mapping model specifically comprises:
acquiring three-dimensional coordinate position information of the wire in the three-dimensional point cloud data according to the wire data mapping model;
and calculating the distance between the three-dimensional coordinate position information of the highest point of the hidden danger target and the coordinate position information of the lead in the three-dimensional point cloud data to obtain the actual distance between the hidden danger target and the lead.
7. The method for measuring the distance of the hidden danger target of the power transmission line channel according to claim 6, wherein after the distance between the hidden danger target and a lead is obtained based on coordinate position information in the three-dimensional point cloud data and a pre-established lead model, the method further comprises:
determining the shortest distance in the actual distances according to the actual distances between the hidden danger target and each point in the wire;
and outputting the threat level of the hidden danger target according to the shortest distance and the preset clearance distances of different voltage levels of the power transmission line.
8. The method according to claim 7, wherein after determining the shortest distance among the actual distances according to the actual distances between the hidden danger target and each point in the wire, the method further comprises:
mapping the conducting wires in the three-dimensional point cloud data through a conducting wire mapping conversion relation so as to obtain a third space mapping conversion relation between the three-dimensional coordinate position information of the conducting wires and the two-dimensional image data;
and mapping the wire point corresponding to the shortest distance to two-dimensional image data according to the third space mapping conversion relation, and marking the wire point in the two-dimensional image.
9. A power transmission line channel hidden danger target ranging device comprises: at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
10. A non-volatile storage medium having stored thereon computer-executable instructions, the executable instructions comprising:
acquiring two-dimensional image data and three-dimensional point cloud data of a power transmission line channel; wherein the two-dimensional image is obtained by shooting by a monocular camera;
detecting a hidden danger target for the two-dimensional image data through a preset target identification segmentation network model to obtain data information of the hidden danger target; wherein, the data information of the hidden danger target at least comprises: the position information of the hidden danger target and the segmentation outline of the hidden danger target;
if the hidden danger target is determined to be in a pre-established channel three-dimensional protection area according to the position information of the hidden danger target, segmenting the hidden danger target according to the segmentation outline of the hidden danger target, and determining the position information of the segmented hidden danger target;
mapping the ground data in the three-dimensional point cloud data through a pre-established ground data mapping model to obtain a first space mapping conversion relation between the ground data and two-dimensional image data; converting the position information of the segmented hidden danger target into three-dimensional coordinate position information according to the first space mapping conversion relation;
acquiring the actual distance between the hidden danger target and a lead based on the three-dimensional coordinate position information of the hidden danger target and a pre-established lead data mapping model; wherein the wire data mapping model comprises three-dimensional coordinate position information of the wire in the three-dimensional point cloud data.
CN202110643955.9A 2021-06-09 2021-06-09 Method, equipment and medium for measuring potential hazards of transmission line channel target Active CN113345019B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110643955.9A CN113345019B (en) 2021-06-09 2021-06-09 Method, equipment and medium for measuring potential hazards of transmission line channel target

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110643955.9A CN113345019B (en) 2021-06-09 2021-06-09 Method, equipment and medium for measuring potential hazards of transmission line channel target

Publications (2)

Publication Number Publication Date
CN113345019A true CN113345019A (en) 2021-09-03
CN113345019B CN113345019B (en) 2023-07-18

Family

ID=77475818

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110643955.9A Active CN113345019B (en) 2021-06-09 2021-06-09 Method, equipment and medium for measuring potential hazards of transmission line channel target

Country Status (1)

Country Link
CN (1) CN113345019B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808096A (en) * 2021-09-14 2021-12-17 成都主导软件技术有限公司 Non-contact bolt looseness detection method and system
CN113822249A (en) * 2021-11-23 2021-12-21 山东信通电子股份有限公司 Method and equipment for monitoring position of hidden danger of overhead line
CN114066985A (en) * 2022-01-11 2022-02-18 深圳金三立视频科技股份有限公司 Method for calculating hidden danger distance of power transmission line and terminal
CN114543680A (en) * 2022-02-18 2022-05-27 国网湖北省电力有限公司电力科学研究院 On-site monitoring and distance measuring method for overhead transmission line channel construction vehicle
CN114820526A (en) * 2022-04-28 2022-07-29 广东电网有限责任公司 Detection management method, device and system for construction hidden danger of power transmission line
CN115014216A (en) * 2022-06-27 2022-09-06 山东信通电子股份有限公司 Method and device for detecting icing of power transmission line
CN116027321A (en) * 2022-01-14 2023-04-28 北京中创恒益科技有限公司 High-precision real-time protection system and method for power transmission line
CN116091494A (en) * 2023-04-07 2023-05-09 智洋创新科技股份有限公司 Method for measuring distance of hidden danger of external damage of power transmission machinery
CN116245932A (en) * 2023-05-10 2023-06-09 智洋创新科技股份有限公司 Hidden danger ranging method for power transmission line based on grid idea
CN117092641A (en) * 2023-10-20 2023-11-21 江苏翰林正川工程技术有限公司 Information fusion method based on radar ranging and image recognition
WO2024099431A1 (en) * 2022-11-11 2024-05-16 华为技术有限公司 Calibration method based on point cloud data and image, and related apparatus
CN118097565A (en) * 2024-04-19 2024-05-28 国网上海市电力公司 Method for judging invalid hidden trouble of power transmission channel based on monocular vision

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107391631A (en) * 2017-07-10 2017-11-24 国家电网公司 A kind of electric transmission line channel solid space monitoring and fast ranging method
CN110232418A (en) * 2019-06-19 2019-09-13 深圳前海达闼云端智能科技有限公司 Semantic recognition method, terminal and computer readable storage medium
CN110264416A (en) * 2019-05-28 2019-09-20 深圳大学 Sparse point cloud segmentation method and device
CN111353726A (en) * 2020-04-07 2020-06-30 国网福建省电力有限公司漳州供电公司 Power transmission and distribution line passageway defect hidden danger analysis and processing system
CN112396650A (en) * 2020-03-30 2021-02-23 青岛慧拓智能机器有限公司 Target ranging system and method based on fusion of image and laser radar
CN112489130A (en) * 2021-01-11 2021-03-12 众芯汉创（北京）科技有限公司 Distance measuring method and device for power transmission line and target object and electronic equipment
CN112581629A (en) * 2020-12-09 2021-03-30 中国科学院深圳先进技术研究院 Augmented reality display method and device, electronic equipment and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107391631A (en) * 2017-07-10 2017-11-24 国家电网公司 A kind of electric transmission line channel solid space monitoring and fast ranging method
CN110264416A (en) * 2019-05-28 2019-09-20 深圳大学 Sparse point cloud segmentation method and device
CN110232418A (en) * 2019-06-19 2019-09-13 深圳前海达闼云端智能科技有限公司 Semantic recognition method, terminal and computer readable storage medium
CN112396650A (en) * 2020-03-30 2021-02-23 青岛慧拓智能机器有限公司 Target ranging system and method based on fusion of image and laser radar
CN111353726A (en) * 2020-04-07 2020-06-30 国网福建省电力有限公司漳州供电公司 Power transmission and distribution line passageway defect hidden danger analysis and processing system
CN112581629A (en) * 2020-12-09 2021-03-30 中国科学院深圳先进技术研究院 Augmented reality display method and device, electronic equipment and storage medium
CN112489130A (en) * 2021-01-11 2021-03-12 众芯汉创（北京）科技有限公司 Distance measuring method and device for power transmission line and target object and electronic equipment

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808096A (en) * 2021-09-14 2021-12-17 成都主导软件技术有限公司 Non-contact bolt looseness detection method and system
CN113808096B (en) * 2021-09-14 2024-01-30 成都主导软件技术有限公司 Non-contact bolt loosening detection method and system
CN113822249A (en) * 2021-11-23 2021-12-21 山东信通电子股份有限公司 Method and equipment for monitoring position of hidden danger of overhead line
CN114066985A (en) * 2022-01-11 2022-02-18 深圳金三立视频科技股份有限公司 Method for calculating hidden danger distance of power transmission line and terminal
CN116027321B (en) * 2022-01-14 2024-01-30 北京中创恒益科技有限公司 High-precision real-time protection system and method for power transmission line
CN116027321A (en) * 2022-01-14 2023-04-28 北京中创恒益科技有限公司 High-precision real-time protection system and method for power transmission line
CN114543680A (en) * 2022-02-18 2022-05-27 国网湖北省电力有限公司电力科学研究院 On-site monitoring and distance measuring method for overhead transmission line channel construction vehicle
CN114543680B (en) * 2022-02-18 2022-12-02 国网湖北省电力有限公司电力科学研究院 On-site monitoring and distance measuring method for overhead transmission line channel construction vehicle
CN114820526A (en) * 2022-04-28 2022-07-29 广东电网有限责任公司 Detection management method, device and system for construction hidden danger of power transmission line
CN115014216A (en) * 2022-06-27 2022-09-06 山东信通电子股份有限公司 Method and device for detecting icing of power transmission line
WO2024099431A1 (en) * 2022-11-11 2024-05-16 华为技术有限公司 Calibration method based on point cloud data and image, and related apparatus
CN116091494A (en) * 2023-04-07 2023-05-09 智洋创新科技股份有限公司 Method for measuring distance of hidden danger of external damage of power transmission machinery
CN116245932B (en) * 2023-05-10 2023-08-22 智洋创新科技股份有限公司 Hidden danger ranging method for power transmission line based on grid idea
CN116245932A (en) * 2023-05-10 2023-06-09 智洋创新科技股份有限公司 Hidden danger ranging method for power transmission line based on grid idea
CN117092641B (en) * 2023-10-20 2023-12-15 江苏翰林正川工程技术有限公司 Information fusion method based on radar ranging and image recognition
CN117092641A (en) * 2023-10-20 2023-11-21 江苏翰林正川工程技术有限公司 Information fusion method based on radar ranging and image recognition
CN118097565A (en) * 2024-04-19 2024-05-28 国网上海市电力公司 Method for judging invalid hidden trouble of power transmission channel based on monocular vision

Also Published As

Publication number Publication date
CN113345019B (en) 2023-07-18

Similar Documents

Publication Publication Date Title
CN113345019B (en) Method, equipment and medium for measuring potential hazards of transmission line channel target
CN112766274B (en) Water gauge image water level automatic reading method and system based on Mask RCNN algorithm
CN107392247B (en) Real-time detection method for ground object safety distance below power line
CN109829908B (en) Binocular image-based method and device for detecting safety distance of ground object below power line
CN113887641A (en) Hidden danger target determination method, device and medium based on power transmission channel
CN113240887A (en) Slope rockfall collapse monitoring method, device and equipment
CN111680574B (en) Face detection method and device, electronic equipment and storage medium
CN114743021A (en) Fusion method and system of power transmission line image and point cloud data
CN113970734B (en) Method, device and equipment for removing snowfall noise points of road side multi-line laser radar
CN114255405A (en) Hidden danger target identification method and device
CN110910379B (en) Incomplete detection method and device
CN116363585A (en) On-line monitoring method and system for power transmission line
CN117372401A (en) Cable line defect identification method, system, terminal equipment and medium
CN115019254A (en) Method, device, terminal and storage medium for detecting foreign matter invasion in power transmission area
WO2024099431A1 (en) Calibration method based on point cloud data and image, and related apparatus
CN114114291A (en) Distance detection method, device, server and storage medium
CN112816053A (en) Non-contact vibration information detection method and system for ship equipment
CN115393387A (en) Building displacement monitoring method and device
JP6546898B2 (en) Three-dimensional space identification apparatus, method, and program
US20230243976A1 (en) Systems and methods for utility pole loading and/or clearance analyses
Zhu et al. Target Measurement Method Based on Sparse Disparity for Live Power Lines Maintaining Robot
CN116973939B (en) Safety monitoring method and device
US20240257376A1 (en) Method and system for detection a line above ground from a helicopter
CN116168283A (en) Method, equipment and storage medium for judging hidden danger degree of transmission line channel
CN110807773B (en) Panoramic image detection method for surface defects of nuclear power station

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant