CN111652163A - Transmission line tower line segment matching method and equipment - Google Patents
Transmission line tower line segment matching method and equipment Download PDFInfo
- Publication number
- CN111652163A CN111652163A CN202010512609.2A CN202010512609A CN111652163A CN 111652163 A CN111652163 A CN 111652163A CN 202010512609 A CN202010512609 A CN 202010512609A CN 111652163 A CN111652163 A CN 111652163A
- Authority
- CN
- China
- Prior art keywords
- matching
- line
- line segment
- dimensional
- line segments
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for matching a pole tower line segment of a power transmission line, wherein the method comprises the following steps: extracting line segments from a tower area in an image of a power transmission line tower, carrying out conditional constraint on intersection points among the line segments to obtain original intersection points, clustering the original intersection points to obtain new intersection points, matching the line segments corresponding to the new intersection points based on a plane optimization and coplanarity cost method, and selecting a matching point of the new intersection points and a matching line segment corresponding to the new intersection points; and selecting a main plane from the planes by fitting the planes where the line segments are located, and searching three-dimensional coordinates of other main planes meeting the approximate collinear condition according to the three-dimensional coordinates in the main plane, thereby obtaining the three-dimensional line segments matched with the power tower. According to the method, a plurality of useless intersection points of the line segments are removed through plane optimization and coplanarity cost methods, and finally, the three-dimensional line segments of the electric power tower are matched according to the approximate collinear condition, so that the line segment matching of the three-dimensional image of the electric power tower is realized in a large number of intersected line segments.
Description
Technical Field
The invention relates to the field of line segment matching, in particular to a method and equipment for matching a line segment of a power transmission line tower.
Background
With the rapid development of information technology, three-dimensional reconstruction has more and more applications in many fields, such as cultural heritage protection, cultural relic research, video monitoring, animation and movie production, medical image processing, virtual reality and the like. In computer vision, the image-based three-dimensional reconstruction technology is a process of automatically calculating and matching two-dimensional geometric information and depth information of a scene by a computer according to two or more than two-dimensional images of the scene, and finally establishing a three-dimensional model. In the traditional three-dimensional reconstruction, the reconstruction precision of a three-dimensional point cloud model in a place lacking texture is poor based on the feature extraction and matching of points, and a three-dimensional line segment model can provide more sufficient structural information and reflect the geometric topological relation of a scene. To generate a three-dimensional reconstruction model based on line segments requires establishing correspondence between two-dimensional line segments from different pictures, and line segment matching refers to a process of establishing correspondence between two sets of line segments. However, the power tower is mainly composed of interconnected steel structures, so that a large number of intersecting line segments appear in a two-dimensional image, and a method for matching the line segments in the three-dimensional image of the power tower is not available at present.
In summary, there is no technical problem in the prior art that a method for performing line segment matching on a three-dimensional image of a power tower is available.
Disclosure of Invention
The invention provides a line segment matching method and equipment for a power transmission line tower, which are used for solving the technical problem that no method for matching a line segment for a three-dimensional image of a power transmission line tower exists in the prior art.
The invention provides a transmission line pole tower line segment matching method, which comprises the following steps:
acquiring power transmission line tower images acquired by an unmanned aerial vehicle, and extracting tower area images from the power transmission line tower images;
extracting line segments from the tower area image, and performing conditional constraint on intersection points among the line segments to obtain all original intersection points;
performing cluster analysis on all the original intersection points to obtain a plurality of cluster categories and cluster centers corresponding to the cluster categories, and taking the cluster centers corresponding to the cluster categories as new intersection points;
merging the line segments corresponding to the new intersection points in each cluster category, and performing plane optimization line segment matching on the line segments corresponding to the merged new intersection points to obtain a plurality of candidate matching points and candidate matching line segments corresponding to the plurality of candidate matching points;
calculating the coplanarity cost from the new intersection point to a plurality of candidate matching points, and selecting the matching point of the new intersection point and the matching line segment corresponding to the new intersection point;
fitting the line segment corresponding to the new intersection point and the plane where the matched line segment corresponding to the new intersection point is located;
traversing the plane where the line segment corresponding to each new intersection point is located, counting the number of the matched line segments corresponding to the new intersection points in each plane, and taking the plane with the largest number of the matched line segments as the correct plane where the line segment corresponding to the new intersection point is located;
selecting a main plane from the correct plane of the line segment corresponding to each new intersection point, and calculating the three-dimensional coordinates corresponding to the line segment in each main plane;
and matching the three-dimensional coordinates corresponding to the line segments in each main plane, and finding out the three-dimensional coordinates of other main planes meeting the approximate collinear condition to obtain the three-dimensional line segments matched with the power tower.
Preferably, the heading overlapping rate of the images acquired by the unmanned aerial vehicle on the power transmission line tower is more than 60%, and the lateral overlapping rate is more than 30%.
Preferably, each line segment is extracted from the tower area by using an LSD algorithm.
Preferably, the conditional constraint is performed on the intersection points between the line segments, and the specific process of obtaining all the original intersection points is as follows:
and for each line segment extracted from the tower area image, in a rectangular area with preset length and width and with the line segment as the center, the intersection points, which are closest to the two end points of the line segment, of all other line segments intersected with the line segment are reserved to obtain all original intersection points.
Preferably, a DBSCAN algorithm is adopted to perform cluster analysis on all the original intersection points.
Preferably, the specific process of performing the line segment matching for the plane optimization to obtain a plurality of candidate matching points and a plurality of corresponding candidate matching line segments is as follows:
calculation of Ii、IjThe epipolar lines of the two transmission line tower images are in the IiSearch and I along the epipolar line direction in the imagejThe distance from the intersection point to the epipolar line is less than IiAnd if the maximum distance from the original intersection point in the image to the combined new intersection point is the maximum, the intersection point is a candidate matching point of the new intersection point, and the line segment corresponding to the intersection point is a candidate matching line segment.
Preferably, the specific process of calculating the coplanarity cost from the new intersection point to the multiple candidate matching points and selecting the matching point of the new intersection point and the corresponding matching line segment is as follows:
calculation of IjCalculating a three-dimensional coordinate point of intersection of the foot point and the candidate matching point by utilizing a collinearity equation from the new intersection point of the image to the foot point of the epipolar line;
given of IjTaking a unit vector of a ray direction in which the image and the new intersection point are located as an initial normal vector, and calculating a three-dimensional coordinate point of a line segment corresponding to the new intersection point based on the initial normal vector and the three-dimensional coordinate point;
projecting the three-dimensional coordinate point of the line segment corresponding to the new intersection point to IjForming two-dimensional line segments in the image;
and taking the point in the two-dimensional line segment with the shortest vertical distance with the new intersection point as the matching point of the new intersection point, and calculating the matching line segment corresponding to the new intersection point through the Hungarian minimum bipartite graph algorithm.
Preferably, the specific process of selecting the main plane from the correct planes of the line segments corresponding to each new intersection point is as follows:
calculating the three-dimensional coordinates corresponding to the two-dimensional line segment in the plane where the line segment corresponding to each new intersection point is located correctly,
projecting the three-dimensional coordinates to other images of the transmission line towers, and searching for two-dimensional matching line segments which meet the constraint in the images of the other transmission line towers;
calculating a three-dimensional line segment corresponding to the two-dimensional matching line segment, and merging three-dimensional planes where the three-dimensional line segments are located;
and calculating the number of the three-dimensional line segments contained in the correct plane of the line segment corresponding to each new intersection point, and selecting a main plane based on the number of the three-dimensional line segments.
Preferably, the specific process of finding out the three-dimensional coordinates of other main planes meeting the approximate collinear condition to obtain the three-dimensional line segment matched with the power tower is as follows:
by searching other three-dimensional coordinates within a certain radius range of the three-dimensional coordinates, if no other three-dimensional coordinates exist within the range, matching is wrong, and the three-dimensional coordinates need to be removed; and if other three-dimensional coordinates exist, judging whether the other three-dimensional coordinates are approximately collinear with the three-dimensional coordinates, and if so, taking the three-dimensional line segment corresponding to the three-dimensional coordinates as the three-dimensional line segment matched with the power tower.
A transmission line tower line segment matching device comprises a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the transmission line tower line segment matching method according to the instructions in the program codes.
According to the technical scheme, the embodiment of the invention has the following advantages:
extracting each line segment from a tower area in an image of a tower of a power transmission line, and performing condition constraint on intersection points between each line segment to obtain all original intersection points;
performing cluster analysis on all the original intersection points to obtain a plurality of cluster categories and cluster centers corresponding to the cluster categories, and taking the cluster centers corresponding to the cluster categories as new intersection points;
merging the line segments corresponding to the new intersection points in each cluster category, and performing plane optimization line segment matching on the line segments corresponding to the merged new intersection points to obtain a plurality of candidate matching points and a plurality of corresponding candidate matching line segments;
calculating the coplanarity cost from the new intersection point to a plurality of candidate matching points, and selecting the matching point of the new intersection point and the matching line segment corresponding to the new intersection point;
fitting the line segment corresponding to the new intersection point and the plane where the matched line segment corresponding to the new intersection point is located;
traversing the plane where the line segment corresponding to each new intersection point is located, counting the number of the matched line segments corresponding to the new intersection points in each plane, and taking the plane with the largest number of the matched line segments as the correct plane where the line segment corresponding to the new intersection point is located;
selecting a main plane from the correct plane of the line segment corresponding to each new intersection point, and calculating the three-dimensional coordinates corresponding to the line segment in each main plane;
and matching the three-dimensional coordinates corresponding to the line segments in each main plane, and finding out the three-dimensional coordinates of other main planes meeting the approximate collinear condition to obtain the three-dimensional line segments matched with the power tower.
Extracting line segments from a tower area in an image of a power transmission line tower, performing conditional constraint on intersection points among the line segments to obtain all original intersection points, clustering the original intersection points, taking a clustering center corresponding to each clustering category as a new intersection point, matching line segments corresponding to the new intersection points based on a plane optimization and coplanarity cost method, and selecting a matching point of the new intersection point and a matching line segment corresponding to the new intersection point; and selecting a main plane from the planes by fitting the planes where the line segments are located, and searching three-dimensional coordinates of other main planes meeting the approximate collinear condition according to the three-dimensional coordinates in the main plane, thereby obtaining the three-dimensional line segments matched with the power tower. According to the embodiment of the invention, a large number of useless line segments in the image are removed by using the constraint condition of the line segment intersection points, a plurality of useless intersection points of the line segments are removed by using a plane optimization and coplanarity cost method, and finally, the three-dimensional line segment matching of the electric power tower is carried out according to an approximately collinear condition, so that the line segment matching of the three-dimensional image of the electric power tower can be realized in a large number of intersected line segments, and the technical problem that a line segment matching method aiming at the three-dimensional image of the electric power tower is not available in the prior art is solved. Therefore, technicians can construct a three-dimensional line segment model of the power transmission line tower according to the image acquired by the unmanned aerial vehicle, and the structural information of the power transmission line tower and the geometric topological relation of the scene are fully obtained from the three-dimensional line segment model, so that whether the structure of the power transmission line tower is normal or not is judged, and the operation safety of the power network is guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a method flowchart of a method and a device for matching a pole tower line segment of a power transmission line provided by an embodiment of the present invention.
Fig. 2 is a schematic diagram of a method and a device for matching line segments of a power transmission line tower according to an embodiment of the present invention for performing conditional constraint on intersections between line segments.
Fig. 3 is a schematic diagram of line segment matching based on planar optimization of a method and a device for matching a line segment of a power transmission line tower provided by an embodiment of the present invention.
Fig. 4 is an equipment framework diagram of a method and equipment for matching a pole tower line segment of a power transmission line provided by an embodiment of the invention.
Detailed Description
The embodiment of the invention provides a line segment matching method and equipment for a power transmission line tower, which are used for solving the technical problem that no method for matching a line segment for a three-dimensional image of a power tower exists in the prior art.
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a method and an apparatus for matching a pole tower line segment of a power transmission line according to an embodiment of the present invention.
The embodiment of the invention provides a method for matching a pole and tower line segment of a power transmission line, which comprises the following steps:
acquiring an electric transmission line tower image acquired by an unmanned aerial vehicle, and extracting a tower area from the electric transmission line tower image by utilizing a deep learning technology, namely fast-RCNN;
extracting line segments from the tower area, and performing conditional constraint on intersection points among the line segments to obtain all original intersection points; because the position relation between straight lines is parallel and intersected in the two-dimensional plane, if the straight lines of all line segments are directly intersected, a large number of intersection points exist, subsequent matching is not facilitated, the calculation conditions of the intersection points of the line segments need to be restricted, and therefore the subsequent calculation amount is reduced.
And performing cluster analysis on all the original intersection points to obtain a plurality of cluster categories and cluster centers corresponding to each cluster category, combining all the intersection points in the categories in each cluster category, and taking the cluster center points of the intersection points as new intersection points.
And combining the line segments corresponding to the new intersection points in each cluster type, wherein each calculated original intersection point records the information of which two line segments the intersection point is calculated from, and when the center point of the cluster is calculated as the new intersection point after the intersection points are clustered, the information of the corresponding line segments recorded by the original intersection point needs to be combined into the attribute of the new intersection point. For example: given the clustered intersection set p1,p2,p3With center point p', p1Is prepared from1And l2Calculated as p2Is prepared from1And l3Calculated as p3Is prepared from3And l4If the calculated result is that the p' is merged with the corresponding line segment set to be l1,l2,l3,l4. After merging the intersection points, calculating the distance from each original intersection point to the new intersection point, and taking the maximum value as the new intersection pointThe radius of the intersection point. Performing plane optimization line segment matching on the line segment corresponding to the combined new intersection point to obtain a plurality of candidate matching points and a plurality of corresponding candidate matching line segments;
calculating the coplanarity cost from the new intersection point to a plurality of candidate matching points, and selecting the matching point of the new intersection point and the matching line segment corresponding to the new intersection point; because each new intersection point has a plurality of candidate intersection points to be matched in other images, a part of candidate intersection points need to be removed through a certain cost function, correct matching points are selected, and subsequent calculation amount is reduced.
Fitting the line segment corresponding to the new intersection point and the plane of the matched line segment corresponding to the new intersection point by using a least square method to prepare for subsequent elimination of the error matched line segment;
traversing the plane where the line segment corresponding to each new intersection point is located, counting the number of the matched line segments corresponding to the new intersection points in each plane, and taking the plane with the largest number of the matched line segments as the correct plane where the line segment corresponding to the new intersection point is located; because the coplanar line segments still keep coplanarity in the correctly matched line segments in different images, the correct matched line segments can be found in the candidate matched line segments matched with the two-dimensional line segments according to the characteristic. After all the intersection points and the corresponding line segments are matched, a large number of wrong matching line segments and plane information can be removed through the steps of traversing the plane where each line segment is located, counting the number of the matching line segments in other images on the plane, taking the plane with the largest number of coplanar line segments as the plane where the line segments are located, and keeping the matched line segment pairs in other images recorded by the plane as correct matching line segments.
Selecting a main plane from the correct plane of the line segment corresponding to each new intersection point, and calculating the three-dimensional coordinates corresponding to the line segment in each main plane;
and matching the three-dimensional coordinates corresponding to the line segments in each main plane, and finding out the three-dimensional coordinates of other main planes meeting the approximate collinear condition to obtain the three-dimensional line segments matched with the power tower. The line segment matching of the power transmission line tower is carried out through two images, the power transmission line tower is visible in a plurality of images, the same three-dimensional line segment in the power transmission line tower can be obtained through pairwise matching of different images, and the three-dimensional line segment obtained through calculation after matching meets the condition of approximate collinearity.
As a preferred embodiment, the heading overlapping rate of the images acquired by the unmanned aerial vehicle on the power transmission line tower is required to be more than 60%, and the side overlapping rate is more than 30%. The course overlapping is to meet the requirement of stereo observation, the side overlapping is to connect the ground objects of adjacent flight paths, the maximum number of the overlapped images with 60% of the flight direction overlapping degree on one flight path is 3, the side overlapping degree is 30%, the number of the overlapped images is 2, and therefore the maximum number of the overlapped images which can be achieved by the aerial photography images is 2 multiplied by 3 to 6.
As a preferred embodiment, an LSD algorithm is used to extract each line segment from the tower area. The LSD algorithm is a straight line extraction algorithm, the main idea is to obtain the gradient of gray scale by derivation, because the vertical direction of the gray scale gradient is the direction of a line, vectors with the same direction are circled by a rectangle, the behavior is refined, and finally each line segment can be obtained, and the specific process is as follows:
1. and performing Gaussian down-sampling on the input power transmission line tower image in a scale of s-0.8.
2. The gradient value and the gradient-line orientation (level-line orientation) of each point are calculated.
3. Pseudo-ordering (pseudo-ordered) all points according to gradient values, building a list of states, all points set to UNUSED.
4. And setting the corresponding position in the point state table with the gradient value smaller than rho as USED.
5. The point with the largest gradient (first position of the pseudo-arrangement) in the list is taken out as the seed point (seed), and the state list is set as USED.
do：
a. With seed as a starting point, search for surrounding UNUSED and points whose direction is within the threshold [ -t, t ] range, the state changes to USED.
b. A rectangle R is generated that contains all the satisfied points.
c. And judging whether the density of the homologous points (alignedpt) meets a threshold value D, and if not, changing the truncation (cut) R into a plurality of rectangular frames until meeting the threshold value D.
d. The NFA is calculated.
e. Changing R makes the value of NFA smaller until NFA < ═ R is added to the output list.
As a preferred embodiment, the specific process of performing conditional constraint on the intersection point between each line segment to obtain all the original intersection points is as follows:
as shown in fig. 2, for a line segment l extracted in a single imageiDefining a length and a width, | l, respectively, centered along the line segmentiA rectangular region R of | +2w and h for a line segment l within the rectangular region RjCalculating liAnd ljAnd only leaving the line segment liThe intersection point where the two end points are closest. The dotted rectangular area in FIG. 2 is represented by1Is a centrally flaring R, wherein2And l3Intersecting the rectangle R, the two segments participate in the intersection calculation, and l4The intersection point calculation is not participated in without intersecting the rectangle R. Wherein l2And l1Has a point of intersection of p1，l3And l1Has a point of intersection of p2And p is1To line segment l1The distance of the end points is less than p2To l1Distance of end points, when only p is reserved1And record p1Is prepared from1And l3The two line segments are calculated to intersect.
As a preferred embodiment, the DBSCAN algorithm is used to perform cluster analysis on all the original intersection points. The DBSCAN algorithm is a relatively representative density-based clustering algorithm, and unlike the partitioning and hierarchical clustering method, the DBSCAN algorithm defines clusters as a maximum set of density-connected points, can partition an area having a sufficiently high density into clusters, and can find clusters of arbitrary shapes in a spatial database of noise. In this embodiment, the two parameters eps and MinPts in DBSCAN are set to 30 and 3 according to the image resolution and experience to ensure a more robust correct line segment matching rate.
As a preferred embodiment, the specific process of performing the line segment matching for plane optimization to obtain a plurality of candidate matching points and a plurality of corresponding candidate matching line segments is as follows:
calculation of Ii、IjThe epipolar lines of the two transmission line tower images are in the IiSearch and I along the epipolar line direction in the imagejThe distance from the intersection point to the epipolar line is less than IiAnd if the maximum distance from the original intersection point in the image to the combined new intersection point is the maximum, the intersection point is a candidate matching point of the new intersection point, and the line segment corresponding to the intersection point is a candidate matching line segment. In the present embodiment, as illustrated in FIG. 3, the following example is provided
Are respectively an image Ii、IjCamera projection center of (1), Ri、RjIts corresponding rotation matrix;
as an image IiThe m-th new intersection point after the calculation is merged, and the radius of the new intersection point is
(maximum distance from the original intersection point of the cluster to the new intersection point after combination), the corresponding two-dimensional line segment set is
The end point coordinates of each line segment are
To be Ci、CjIs taken as a base line
Epipolar line of point calculation by formula
And calculating to obtain the matrix, wherein F is a basic matrix. Given two images Ii、IjAnd an imageIiCross point of (1)
Calculate its corresponding epipolar line
And in the image IjSearching for the intersection point along the direction of the epipolar line, and if the intersection point is found
The distance to the epipolar line is less than the radius of the intersection point, then
Is composed of
Then further calculating the candidate matching points
Is matched to
And judging whether the coplanarity cost meets the coplanarity constraint condition or not.
As a preferred embodiment, the specific process of calculating the coplanarity cost from the new intersection point to the multiple candidate matching points and selecting the matching point of the new intersection point and the corresponding matching line segment is as follows:
calculation of IjNew intersection point of image
To the core line
Foot drop point of
Calculating the point of the foot drop by using the collinearity equation
And candidate matching points
Three-dimensional coordinate points of forward intersection
Given of IjThe unit vector of the ray direction of the image and the new intersection point is used as an initial normal vector, and the three-dimensional coordinate point of the line segment corresponding to the new intersection point is calculated based on the initial normal vector and the three-dimensional coordinate point, wherein the specific process is as follows:
given the initial normal vector of the intersection point
Is CiAnd
unit vector of the direction of the ray, and three-dimensional coordinate point
Suppose that
The corresponding line segments are coplanar, and initialized normal vectors are utilized
And three-dimensional points
Can calculate out
Three-dimensional coordinate points of the corresponding line segments, wherein the endpoints are
Calculate its three-dimensional coordinates
For example, the following steps are carried out:
Wherein
The rotation matrix of the ith image is transposed, and (cx, cy, f) is the image principal point and the image principal distance of the image.
Projecting the three-dimensional coordinate point of the line segment corresponding to the new intersection point to IjForming two-dimensional line segments in the image; the specific process is as follows: projecting three-dimensional line segments formed by three-dimensional coordinate points to an image IjCorresponding two-dimensional line segment in
By minimizing projected line segments
To
Corresponding line segment
Is optimized by the minimum distance
And
optimizing Z of object space point along over-projection central ray directionnThe normal vector is represented by two angles α, β, where:
the minimum objective equation thus obtained is
Wherein
Is composed of
To
The minimum vertical distance of the matched line segment.
And taking the point in the two-dimensional line segment with the shortest vertical distance with the new intersection point as the matching point of the new intersection point, and calculating the matching line segment corresponding to the new intersection point through the Hungarian minimum bipartite graph algorithm. For a given line segment
And
any one line segment
The matching weight calculation function of the Hungarian minimum bipartite graph algorithm is
w＝angle/exp(overlap/(d1+d2+1)
Wherein angle and overlap are the included angle and overlap ratio of the two line segments, d1And d2Is a line segment
Respectively to the line segment
The distance of (c).
It needs to be further explained that: because the transmission line tower is in a hollow structure, line segments of different planes are overlapped in front and back in the imageTherefore, when the matching algorithm is used for matching, any two line segments are adopted
And
optimizing and matching in sequence, and if the pixel difference calculated after optimization is less than 8, determining that the pixel difference is less than 8
And
is matched to in the minimum vertical distance optimization
Middle corresponding line segment
And
are respectively as
And
and make statistics of the candidate matched line segments
In which each line segment is matched to
The maximum number of matched line segments is taken as the possible matched line segment of the line segment according to the number of each line segment in the
The matched line segments calculate the plane on which the line segments lie. If it is
If the number of the middle line segments is less than 3, a plane is directly fitted by using least square; if it is
And if the number of the middle line segments is more than 3, fitting planes by using least squares for every two straight lines, and combining the fitted planes according to the distance from the planes to the planes and the included angle of the normal vector. When the least square fitting plane is utilized, if the average distance from the three-dimensional line segment to the fitting plane is greater than 0.2m, the fitting plane is considered to be failed, and the plane information is not recorded. At the same time, the fitted plane records all three-dimensional line segments corresponding to the plane and which two-dimensional line segments in which the three-dimensional line segments match.
As a preferred embodiment, the specific process of selecting the main plane from the correct planes of the line segments corresponding to each new intersection point is as follows:
and sequentially intersecting the two-dimensional line segments in the correct plane of the line segment corresponding to each new intersection point with the correct plane of the line segment to calculate corresponding three-dimensional coordinates, projecting the three-dimensional coordinates into other images, judging whether two-dimensional matching line segments meeting the relationship exist in the other images by using the overlapping rate and angle constraint (wherein the overlapping rate is greater than 0.25, and the included angle is less than 5 degrees), calculating the three-dimensional line segment coordinates of the two-dimensional matching line segments and the corresponding three-dimensional planes, merging the three-dimensional planes, calculating the number of three-dimensional line segments contained in the correct plane of the line segment corresponding to each new intersection point, sequencing the number of three-dimensional line segments according to the number of the three-dimensional line segments from large to small, and keeping the correct plane of the line segment corresponding to each new intersection point arranged in the first 20 percent as a main.
As a preferred embodiment, the specific process of finding out the three-dimensional coordinates of other main planes that satisfy the approximate collinear condition and obtaining the three-dimensional line segment matched with the power tower is as follows:
by searching other three-dimensional coordinates within a certain radius range of the three-dimensional coordinates, if no other three-dimensional coordinates exist within the range, matching is wrong, and the three-dimensional coordinates need to be removed; and if other three-dimensional coordinates exist, judging whether the other three-dimensional coordinates are approximately collinear with the three-dimensional coordinates, and if so, taking the three-dimensional line segment corresponding to the three-dimensional coordinates as the three-dimensional line segment matched with the power tower.
As shown in fig. 3, an apparatus 30 for matching a line segment of a power transmission line tower includes a processor 300 and a memory 301;
the memory 301 is used for storing a program code 302 and transmitting the program code 302 to the processor;
the processor 300 is configured to execute the steps in the above-mentioned method for matching a tower segment of a power transmission line according to the instructions in the program code 302.
Illustratively, the computer program 302 may be partitioned into one or more modules/units that are stored in the memory 301 and executed by the processor 300 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 302 in the terminal device 30.
The terminal device 30 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 300, a memory 301. Those skilled in the art will appreciate that fig. 4 is merely an example of a terminal device 30 and does not constitute a limitation of terminal device 30 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.
The Processor 300 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf ProgrammaBle Gate Array (FPGA) or other ProgrammaBle logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 301 may be an internal storage unit of the terminal device 30, such as a hard disk or a memory of the terminal device 30. The memory 301 may also be an external storage device of the terminal device 30, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 30. Further, the memory 301 may also include both an internal storage unit and an external storage device of the terminal device 30. The memory 301 is used for storing the computer program and other programs and data required by the terminal device. The memory 301 may also be used to temporarily store data that has been output or is to be output.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for matching pole tower line segments of a power transmission line is characterized by comprising the following steps:
acquiring power transmission line tower images acquired by an unmanned aerial vehicle, and extracting tower area images from the power transmission line tower images;
extracting line segments from the tower area image, and performing conditional constraint on intersection points among the line segments to obtain all original intersection points;
performing cluster analysis on all the original intersection points to obtain a plurality of cluster categories and cluster centers corresponding to the cluster categories, and taking the cluster centers corresponding to the cluster categories as new intersection points;
merging the line segments corresponding to the new intersection points in each cluster category, and performing plane optimization line segment matching on the line segments corresponding to the merged new intersection points to obtain a plurality of candidate matching points and candidate matching line segments corresponding to the plurality of candidate matching points;
calculating the coplanarity cost from the new intersection point to a plurality of candidate matching points, and selecting the matching point of the new intersection point and the matching line segment corresponding to the new intersection point;
fitting the line segment corresponding to the new intersection point and the plane where the matched line segment corresponding to the new intersection point is located;
traversing the plane where the line segment corresponding to each new intersection point is located, counting the number of the matched line segments corresponding to the new intersection points in each plane, and taking the plane with the largest number of the matched line segments as the correct plane where the line segment corresponding to the new intersection point is located;
selecting a main plane from the correct plane of the line segment corresponding to each new intersection point, and calculating the three-dimensional coordinates corresponding to the line segment in each main plane;
and matching the three-dimensional coordinates corresponding to the line segments in each main plane, and finding out the three-dimensional coordinates of other main planes meeting the approximate collinear condition to obtain the three-dimensional line segments matched with the power tower.
2. The method for matching the line segments of the power transmission line towers according to claim 1, wherein the images of the power transmission line towers collected by the unmanned aerial vehicle require a course overlapping rate of the images to be greater than 60% and a side overlapping rate to be greater than 30%.
3. The method for matching the line segments of the transmission line towers according to claim 1, wherein each line segment is extracted from the tower area by using an LSD algorithm.
4. The method for matching the line segments of the transmission line tower according to claim 1, wherein the specific process of obtaining all the original intersection points by performing conditional constraint on the intersection points between the line segments is as follows:
and for each line segment extracted from the tower area image, in a rectangular area with preset length and width and with the line segment as the center, the intersection points, which are closest to the two end points of the line segment, of all other line segments intersected with the line segment are reserved to obtain all original intersection points.
5. The method for matching the line segments of the transmission line towers according to claim 1, wherein a DBSCAN algorithm is adopted to perform cluster analysis on all the original intersection points.
6. The method for matching the line segments of the power transmission line towers according to claim 1, wherein the specific process of performing the line segment matching for plane optimization to obtain a plurality of candidate matching points and a plurality of corresponding candidate matching line segments comprises the following steps:
calculation of Ii、IjThe epipolar lines of the two transmission line tower images are in the IiSearch and I along the epipolar line direction in the imagejThe distance from the intersection point to the epipolar line is less than IiAnd if the maximum distance from the original intersection point in the image to the combined new intersection point is the maximum, the intersection point is a candidate matching point of the new intersection point, and the line segment corresponding to the intersection point is a candidate matching line segment.
7. The method for matching the line segments of the power transmission line towers according to claim 6, wherein the specific process of calculating the coplanarity cost from the new intersection point to the candidate matching points and selecting the matching points of the new intersection point and the corresponding matching line segments comprises the following steps:
calculation of IjCalculating a three-dimensional coordinate point of intersection of the foot point and the candidate matching point by utilizing a collinearity equation from the new intersection point of the image to the foot point of the epipolar line;
given of IjTaking a unit vector of a ray direction in which the image and the new intersection point are located as an initial normal vector, and calculating a three-dimensional coordinate point of a line segment corresponding to the new intersection point based on the initial normal vector and the three-dimensional coordinate point;
projecting the three-dimensional coordinate point of the line segment corresponding to the new intersection point to IjForming two-dimensional line segments in the image;
and taking the point in the two-dimensional line segment with the shortest vertical distance with the new intersection point as the matching point of the new intersection point, and calculating the matching line segment corresponding to the new intersection point through the Hungarian minimum bipartite graph algorithm.
8. The method for matching the line segments of the power transmission line towers according to claim 7, wherein the specific process of selecting the main plane from the planes where the line segments corresponding to each new intersection point are correctly located is as follows:
calculating the three-dimensional coordinates corresponding to the two-dimensional line segment in the plane where the line segment corresponding to each new intersection point is located correctly,
projecting the three-dimensional coordinates to other images of the transmission line towers, and searching for two-dimensional matching line segments which meet the constraint in the images of the other transmission line towers;
calculating a three-dimensional line segment corresponding to the two-dimensional matching line segment, and merging three-dimensional planes where the three-dimensional line segments are located;
and calculating the number of the three-dimensional line segments contained in the correct plane of the line segment corresponding to each new intersection point, and selecting a main plane based on the number of the three-dimensional line segments.
9. The method for matching the line segment of the power transmission line tower as claimed in claim 1, wherein the specific process of finding out the three-dimensional coordinates of other main planes which meet the approximate collinear condition to obtain the matched three-dimensional line segment of the power transmission line tower is as follows:
by searching other three-dimensional coordinates within a certain radius range of the three-dimensional coordinates, if no other three-dimensional coordinates exist within the range, matching is wrong, and the three-dimensional coordinates need to be removed; and if other three-dimensional coordinates exist, judging whether the other three-dimensional coordinates are approximately collinear with the three-dimensional coordinates, and if so, taking the three-dimensional line segment corresponding to the three-dimensional coordinates as the three-dimensional line segment matched with the power tower.
10. A transmission line tower line segment matching device is characterized by comprising a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the method for matching a pole tower line segment of a power transmission line according to claims 1 to 9 according to instructions in the program code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010512609.2A CN111652163B (en) | 2020-06-08 | 2020-06-08 | Transmission line tower line segment matching method and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010512609.2A CN111652163B (en) | 2020-06-08 | 2020-06-08 | Transmission line tower line segment matching method and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111652163A true CN111652163A (en) | 2020-09-11 |
| CN111652163B CN111652163B (en) | 2021-08-27 |
Family
ID=72349057
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010512609.2A Active CN111652163B (en) | 2020-06-08 | 2020-06-08 | Transmission line tower line segment matching method and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111652163B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112927388A (en) * | 2021-01-25 | 2021-06-08 | 深圳市广程杰瑞科技有限公司 | Vehicle management method and system for logistics enterprise |
| CN116246038A (en) * | 2023-05-11 | 2023-06-09 | 西南交通大学 | Multi-view three-dimensional line segment reconstruction method, system, electronic equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714530A (en) * | 2012-09-28 | 2014-04-09 | 深圳先进技术研究院 | Vanishing point detection and image correction method |
| CN104091168A (en) * | 2014-07-18 | 2014-10-08 | 深圳供电局有限公司 | Method for automatically extracting and locating power line based on unmanned aerial vehicle image |
| US9536320B1 (en) * | 2014-12-23 | 2017-01-03 | John H. Prince | Multiple coordinated detectors for examination and ranging |
| CN107784652A (en) * | 2017-10-30 | 2018-03-09 | 广东电网有限责任公司机巡作业中心 | A kind of shaft tower quick determination method based on unmanned plane image |
| CN108830933A (en) * | 2018-06-26 | 2018-11-16 | 广东电网有限责任公司 | A kind of electric force pole tower tower body method for reconstructing, system, medium and equipment |
-
2020
- 2020-06-08 CN CN202010512609.2A patent/CN111652163B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714530A (en) * | 2012-09-28 | 2014-04-09 | 深圳先进技术研究院 | Vanishing point detection and image correction method |
| CN104091168A (en) * | 2014-07-18 | 2014-10-08 | 深圳供电局有限公司 | Method for automatically extracting and locating power line based on unmanned aerial vehicle image |
| US9536320B1 (en) * | 2014-12-23 | 2017-01-03 | John H. Prince | Multiple coordinated detectors for examination and ranging |
| CN107784652A (en) * | 2017-10-30 | 2018-03-09 | 广东电网有限责任公司机巡作业中心 | A kind of shaft tower quick determination method based on unmanned plane image |
| CN108830933A (en) * | 2018-06-26 | 2018-11-16 | 广东电网有限责任公司 | A kind of electric force pole tower tower body method for reconstructing, system, medium and equipment |
Non-Patent Citations (4)
| Title |
|---|
| MANUEL HOFER ET AL: "Efficient 3D scene abstraction using line segments", 《COMPUTER VISION AND IMAGE UNDERSTANDING》 * |
| WEI HUANG ET AL: "A Model-Driven Method for Pylon Reconstruction from Oblique UAV Images", 《SENSORS》 * |
| 王志 等: "基于单目视觉和振动补偿的接触线高度及拉出值测量", 《电气化铁道》 * |
| 陈庭旺 等: "面向大规模场景的三维线段检测算法", 《计算机辅助设计与图形学学报》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112927388A (en) * | 2021-01-25 | 2021-06-08 | 深圳市广程杰瑞科技有限公司 | Vehicle management method and system for logistics enterprise |
| CN112927388B (en) * | 2021-01-25 | 2023-08-15 | 广东卓沃供应链管理有限公司 | Logistics enterprise vehicle management method and system |
| CN116246038A (en) * | 2023-05-11 | 2023-06-09 | 西南交通大学 | Multi-view three-dimensional line segment reconstruction method, system, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111652163B (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111369681B (en) | Three-dimensional model reconstruction method, device, equipment and storage medium | |
| JP5797789B2 (en) | Method and system for quasi-duplicate image retrieval | |
| US9942535B2 (en) | Method for 3D scene structure modeling and camera registration from single image | |
| WO2015135323A1 (en) | Camera tracking method and device | |
| WO2021143935A1 (en) | Detection method, device, electronic apparatus, and storage medium | |
| EP3274964B1 (en) | Automatic connection of images using visual features | |
| CN110363817B (en) | Target pose estimation method, electronic device, and medium | |
| CN112927363B (en) | Voxel map construction method and device, computer readable medium and electronic equipment | |
| CN111652163B (en) | Transmission line tower line segment matching method and equipment | |
| CN114219855A (en) | Point cloud normal vector estimation method and device, computer equipment and storage medium | |
| CN110567441A (en) | Particle filter-based positioning method, positioning device, mapping and positioning method | |
| CN112613107A (en) | Method and device for determining construction progress of tower project, storage medium and equipment | |
| Bao et al. | Robust tightly-coupled visual-inertial odometry with pre-built maps in high latency situations | |
| CN115018999A (en) | Multi-robot-cooperation dense point cloud map construction method and device | |
| CN113592015A (en) | Method and device for positioning and training feature matching network | |
| CN112270748A (en) | Three-dimensional reconstruction method and device based on image | |
| US20230053952A1 (en) | Method and apparatus for evaluating motion state of traffic tool, device, and medium | |
| Phalak et al. | DeepPerimeter: Indoor boundary estimation from posed monocular sequences | |
| Geng et al. | SANet: A novel segmented attention mechanism and multi-level information fusion network for 6D object pose estimation | |
| CN113743533B (en) | Picture clustering method and device and storage medium | |
| CN115439534A (en) | Image feature point matching method, device, medium, and program product | |
| CN112257666B (en) | Target image content aggregation method, device, equipment and readable storage medium | |
| CN114723883A (en) | Three-dimensional scene reconstruction method, device, equipment and storage medium | |
| CN114817595A (en) | Sketch-based three-dimensional model retrieval method, device, equipment and medium | |
| CN113763468A (en) | Positioning method, device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |