CN117611740A - Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment - Google Patents

Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment Download PDF

Info

Publication number
CN117611740A
CN117611740A CN202311564747.5A CN202311564747A CN117611740A CN 117611740 A CN117611740 A CN 117611740A CN 202311564747 A CN202311564747 A CN 202311564747A CN 117611740 A CN117611740 A CN 117611740A
Authority
CN
China
Prior art keywords
tower
power
dimensional reconstruction
image
airborne
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311564747.5A
Other languages
Chinese (zh)
Inventor
李亮
赵中奇
魏忠
罗毅
刘权琦
刘泓乐
杨关旭
胡春华
陈凯
莫志武
夜勇
王�锋
赵荣全
韩雪梅
祁小林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Haibei Power Supply Company State Grid Qinghai Electric Power Co ltd
State Grid Qinghai Electric Power Co Ltd
Original Assignee
Haibei Power Supply Company State Grid Qinghai Electric Power Co ltd
State Grid Qinghai Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Haibei Power Supply Company State Grid Qinghai Electric Power Co ltd, State Grid Qinghai Electric Power Co Ltd filed Critical Haibei Power Supply Company State Grid Qinghai Electric Power Co ltd
Priority to CN202311564747.5A priority Critical patent/CN117611740A/en
Publication of CN117611740A publication Critical patent/CN117611740A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
    • H02J13/00002—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by monitoring
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Graphics (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Power Engineering (AREA)
  • Geometry (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

According to the invention, based on the edge vision model guiding mode, the shooting point position is determined through the target recognition mode, so that the unmanned aerial vehicle autonomous inspection in the GPS mode is a new operation mode. The invention can develop new research fields in the aspect of theory and algorithm in the application of artificial intelligence in the power grid, lead new technical directions and promote the innovation and innovation of the related fields of line inspection.

Description

Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment
Technical Field
The invention belongs to the field of electric power construction, and particularly relates to a monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment
Background
The Qinghai province is in Qinghai-Tibet plateau, high in topography, complex in topography, mostly in mountains and ravines, cold in climate and accumulated snow in most areas throughout the year. Meanwhile, qinghai province has rich clean energy, and is an important power output place in Western electric east delivery engineering. In actual electric power operation and maintenance work, because the land is thin, the lines are numerous, and the ultra-high voltage line inspection pressure is high, the lines are easily affected by external force damage, geological disasters, tree barriers, mechanical tension and the like, and the electric power inspection is in face of the problems of personnel shortage, high personal safety risk, low line inspection efficiency quality, incomplete hidden trouble inspection and the like. The intelligent inspection mode represented by the unmanned aerial vehicle has good autonomy and high quality, greatly improves the safety and reliability of inspection, greatly promotes the inspection of the unmanned aerial vehicle in Qinghai in recent years, and simultaneously explores a new operation mode from manual flight to autonomous flight.
At present, no matter the unmanned aerial vehicle autonomous inspection mode is manual teaching or the laser point cloud mainly relies on high-precision positioning to carry out route planning, absolute coordinate position recording of a flight navigation point can be realized based on an RTK technology, and thus, a route can be accurately recorded, and the flying is completed, so that the full autonomous flight of the unmanned aerial vehicle is realized.
The fully autonomous flight mode has the following problems:
(1) Areas without RTKs cannot be patrolled autonomously. The Qinghai province is in Qinghai-Tibet plateau, high in topography, complex in topography, mostly high mountains and ravines in the environment, cold in climate, accumulated snow in most areas throughout the year, and most of current transmission lines are located in mountain areas or remote areas, which are often poor in network signals, and meanwhile, RTK base stations are not available, so that unmanned aerial vehicle autonomous inspection cannot be conducted in an RTK-based mode.
(2) Autonomous inspection depends on the positioning precision of the RTK, and an unmanned aerial vehicle autonomously inspects whether a shooting target meets the specification or not, and depends on whether the navigation point position of each unmanned aerial vehicle autonomously flies or not, if the positioning precision deviates under different weather environment conditions, the shooting target may deviate from the center of an image or the shooting target is unclear.
The image angle shot by adopting the autonomous inspection mode is fixed, and if the route during planning is unreasonable, the later adjustment is complex and inflexible.
Disclosure of Invention
According to the invention, based on the edge vision model guiding mode, the shooting point position is determined through the target recognition mode, so that the unmanned aerial vehicle autonomous inspection in the GPS mode is a new operation mode. The invention can develop new research fields in the aspect of theory and algorithm in the application of artificial intelligence in the power grid, lead new technical directions and promote the innovation and innovation of the related fields of line inspection.
On the popularization way, the dynamic programming algorithm is developed, the dynamic programming algorithm can be applied to edge computing equipment, the dynamic programming algorithm can be transferred to a mobile terminal in the later period, the adopted edge computing equipment can be adapted to main unmanned aerial vehicle models such as Xinjiang, and the like, and the dynamic programming algorithm can be quickly adapted to the existing unmanned aerial vehicle of a power distribution network. On the other hand, the method is closely matched with business departments of all levels of electric power companies in the national network, such as maintenance companies, power supply companies and the like of all levels of electric power companies, test verification is carried out on actual running lines in the research period, and algorithm and application verification is carried out under different tower types in different areas.
The monocular vision real-time three-dimensional reconstruction method based on the airborne edge equipment is characterized by comprising the following specific steps of:
when the digital modeling of the power line corridor is carried out, decomposing and assembling a design standardized model to decompose a pole tower into a tower crown, a tower foundation and an electric element assembly, and then modeling the assembly type of the power transmission line pole tower model;
analyzing the electrical characteristics, the dimensional configuration and the position information of each power tower component to establish an attribute set of the power tower component, and effectively and accurately describing and recording key characteristics of the power tower component;
in order to ensure seamless splicing among the tower crown, the tower foundation and the electrical components, association relations and constraint rules among different types of components in the model library also need to be defined in detail; the method comprises the steps of spatial position constraint, size constraint and functional constraint, and establishing an association relation model and a corresponding constraint rule of connection among all components of a corresponding power tower;
constructing a power pole tower component knowledge base; constructing an inference rule of the power pole tower component to detect whether the corresponding component body meets the rule requirements such as consistency, inclusion relation among classes and the like; the correctness of the electric power tower model constructed by the components can be detected, the types of various electric power towers can be automatically identified, and data information such as correct and reliable configuration, position and the like can be provided for inspection of the electric power towers;
a direct method is adopted at the stage of gentle pose change; the pose changes quickly, and a feature-based method is adopted; the class of the points is selected as the characteristic points or the edge points at one time according to the threshold value by calculating the points of the Shi-Tomas corner points, and the three-dimensional reconstruction of the power line is finally realized by combining with the airborne edge equipment through the comparative research of the algorithm.
The construction of the power tower assembly knowledge base specifically comprises (a) analyzing important terms in the power tower assembly; (b) defining power tower component classes and class hierarchy; (c) defining attributes for each power tower component class; (d) defining facets of the power tower assembly attribute; (e) creating a corresponding power tower assembly instance.
When three-dimensional reconstruction is carried out, the calculation method combining the direct method and the indirect method is as follows:
set G x,y The gray level change amount when the neighborhood window W of the pixel point (x, y) moves (u, v) is obtained by performing diagonalization processing and quadratic calculation according to Talyor series expansion when the movement (u, v) is a local minimum:
wherein I is the image gray level, I x 、I y First partial derivatives in x and y directions respectively,covariance matrix for image gradient, +.>Window being window WFunction, active noise reduction smoothing, lambda 1 、λ 2 The characteristic value of the matrix M is represented by R, which is a two-dimensional rotation matrix;
λ 1 、λ 2 the characteristic value indicates the distribution condition of the gray value of the image pixel; when lambda is 1 、λ 2 When the image window is smaller, no obvious gray level change exists when the image window moves in any direction, which indicates that the corresponding pixel point in the image window is in a smooth area, when lambda 1 、λ 2 When one of the two pixels is bigger and the other is smaller, the gray level change of the image window in one direction is more obvious, and the gray level change in the other direction is not obvious, which means that the corresponding pixel point in the image window is positioned at the boundary of the object, when lambda is 1 、λ 2 When the curvature or the gray gradient change of the pixel point in the image window moving along any direction is larger, the point is the required characteristic point.
When extracting the characteristic points of a whole image, calculating the characteristic values of all pixel points in the image, wherein the calculated amount is relatively large; since the matrix M is a real symmetric matrix, set:
then:
where tr (M) is the trace of matrix M and det (M) is its determinant; to improve computational efficiency, λ is generally avoided by tr (M) and det (M) 1 、λ 2 Is calculated; thereby, corner feature detection functions are obtained:
R(x,y)＝det(M)-k(tr(M)) 2 ＝(AB-C 2 )-k(A+B) 2
here, k=0.04-0.06, which is an empirical constant, may be adaptively modified according to actual scenes.
Given a threshold T, for a pixel point (x, y), when R (x, y) is greater than T and the maximum value in the local neighborhood window, the pixel point (x, y) is the corner point.
If the smaller one of the two characteristic values is larger than the minimum threshold value, a strong angular point is obtained; namely:
R(x,y)＝min(λ 1 ,λ 2 )
since a larger uncertainty depends on smaller feature values, good feature points can be found by finding the maximum of the small feature values.
The beneficial effects are as follows:
(1) A large number of three-dimensional point cloud libraries are constructed in the prior art based on the RTK scheme, point cloud information can be fully excavated based on the prior three-dimensional point cloud libraries, and the method of transfer learning can be adopted to transfer to a GPS mode for full-automatic inspection.
(2) At present, some edge computing devices have low power consumption and can be mounted on unmanned aerial vehicle devices so as to meet the requirements of edge computing when the unmanned aerial vehicle flies, and the device can be applied to the fields of indoor visual navigation, automatic driving, unmanned aerial vehicle obstacle avoidance and the like by researching a real-time three-dimensional reconstruction technology,
(3) The flight mission can be completed by flexibly planning the route based on the current scene by utilizing the real-time three-dimensional vision technology. According to the scheme, the attitude angle of the photographed key part can be determined by utilizing a real-time reconstruction technology of the tower under the condition of no RTK through machine vision research, so that the unmanned aerial vehicle is guided to a proper angle position for photographing.
Drawings
Fig. 1 is a simplified diagram of the AlexNet convolutional network model structure.
FIG. 2 is a technical roadmap for modeling research of heterogeneous power tower components.
Fig. 3 shows a different configuration of the crown assembly.
FIG. 4 tower foundation assemblies of different heights.
Fig. 5 is a direct method diagram.
Fig. 6 is a diagram of a feature-based approach.
Fig. 7 is a window sliding schematic.
FIG. 8λ 1 、λ 2 And the distribution of pixels.
FIG. 9 is a block diagram of a classical Harris algorithm.
Detailed Description
According to the unmanned aerial vehicle autonomous inspection technology for the high-altitude non-RTK signal area based on edge visual recognition, edge computing equipment is carried on an unmanned aerial vehicle, a power transmission equipment target recognition and real-time three-dimensional reconstruction modeling based on edge computing can dynamically generate a route for a high-sea wave non-RTK signal area, the unmanned aerial vehicle is guided to independently inspect and fly through real-time analysis of an edge equipment recognition result, finally, an inspected photo is provided with part position positioning, automatic renaming can be achieved, and meanwhile, defect recognition positioning can be performed by considering a map relation.
Convolutional neural networks can be seen as feature extractors with strong feature learning and feature expression capabilities specifically designed for two-dimensional input data, comprising multiple layers of different network structure layers with weight sharing by local connection, wherein convolutional layers (convolutional layer), pooling layers (pooling layers), activation layers (activation layers), full-connection layers (fully connected layer) and the like are common. Taking AlexNet as an example, which takes the first hand in an image net classification competition in 2012, the AlexNet is an 8-layer convolution network model comprising 5 convolution layers and 2 full-connection layers and an additional input layer, wherein the input layer is a true color image with the size of 227x227 and comprises three channels, a 3-layer window with the size of 11x11 is defined for realizing local sharing weight with the previous layer to become a filter, the filter slides on the input layer by taking 4 pixels as step sizes to carry out convolution operation, and the convolution layer 1 is a 96-dimension 55x 55-size feature layer (feature map) generated after the convolution operation is carried out on the input layer by adopting 96 filters. The feature layer is then typically activated with a specific activation function (activation function), such as ReLU, leak ReLU, etc., that mimics the principle of human neuronal activation, causing some features to propagate forward. In addition, because the image has a "static" property, which also means that features that are useful in one image region are most likely to be equally applicable in another region, so to describe a large image, aggregate statistics can be performed on features at different locations, sliding a window of mxn size over the feature map, and calculating the maximum (or average) of the convolution features within the window, which operation is referred to as a pooling operation. The convolution layer 1 is subjected to maximum pooling operation by adopting a 5x5 window to obtain a pooled layer as shown in fig. 1, then the classical convolution-activation-pooling operation is continuously carried out n times, the final feature layer is fully connected with a high-dimensional vector to complete a classification task, two 2048-dimensional feature vectors are used as shown in fig. 1, and 1000 classes of targets are classified by softmax.
Under the trend of fine and intelligent management of the power grid, the high-precision modeling requirement of the towers is increasingly strong. At present, three-dimensional models of power transmission line towers at home and abroad are mostly manually created by adopting CAD software by referring to corresponding design parameters by professional modeling staff, so that the three-dimensional models are strong in specialization, large in workload, time-consuming and labor-consuming, and difficult to meet the rapid digital modeling of a large number of various heterogeneous power towers. On the basis of analyzing the heterogeneous power tower components, the invention establishes the power tower model based on the ontology technology, and provides accurate information for the identification of the power tower and the positioning of key check points.
(1) Basic component for researching and analyzing electric power pole tower according to electric power pole tower design standard
As shown in fig. 3 and fig. 4, the design and construction of all the transmission line towers must strictly refer to specific tower type standards, and the tower feet of most towers and the tower heads of the similar towers have extremely high similarity, so that the modular modeling of the transmission line tower model can be realized by reasonably decomposing and flexibly assembling the design standardized model when the digital modeling of the power line corridor is performed. For example, the tower can be decomposed into components such as a tower crown, a tower foundation, an electric element and the like.
(2) Constructing a set of attributes for a power tower assembly
And analyzing key characteristics of each power tower assembly to establish an attribute set of the power tower assembly, and effectively and accurately describing and recording the key characteristics of the power tower assembly. Such as: electrical characteristics, dimensional configuration, positional information, and the like.
(3) Analyzing association relationships and constraint rules between components
In order to ensure seamless splicing among the tower crown, the tower foundation and the electrical components, the association relationship and constraint rules among different types of components in the model library also need to be defined in detail. And establishing an association relation model and corresponding constraint rules (mainly comprising spatial position constraint, size constraint, functional constraint and the like) of the connection among all components of the corresponding power tower.
(4) Building power pole tower assembly knowledge base based on ontology
The ontology is a clear standardized description of the conceptual model, and modeling of the power tower assembly, recognition and reasoning of the power tower structure and the like can be realized through the ontology technology. The body has the following elements:
(1) Concept (Concept). Also known as Class (Class).
(2) (2) relationship (Relation). Relationships are used to describe relationships between concepts in an ontology, where typical binary relationships are, for example, is-a relationships between concepts.
(3) Properties (Property). An attribute refers to a predicate among sentences used to describe an Individual (indivisual). The attributes are classified into Object attributes (Object properties) and Data attributes (Data properties). The object attribute connects the individual and the individual, while the data attribute connects the individual and the literal value.
(4) Axiom (Axiom). Axiom in ontology refers to a recognized fact that is used to make knowledge reasoning. (5) Function (Function). A function is a specific expression of a relationship. The mapping relationships specified in the functions may enable reasoning to be directed from one concept to another.
(6) Individual instances of concepts (Individual Instance of Concept). An individual instance of a concept is the lowest logical level concept, and its extension is itself only, simply called the instance.
The construction process of the power pole tower component knowledge base based on the ontology is as follows:
(a) Analyzing important terms in the power tower assembly;
(b) Defining power tower component classes and class hierarchy;
(c) Defining attributes of each power tower component class;
(d) Defining a facet of the power tower component attribute;
(e) A corresponding power tower assembly instance is created.
An ontology's inference engine (Reasoner) can infer logical results from a set of asserted facts or axioms, which play an important role in writing an ontology using OWL. The inference rules of constructing the power tower components are researched to detect whether the corresponding component bodies meet the rule requirements of consistency, inclusion relations among classes and the like. Therefore, the correctness of the electric power tower model constructed by the components can be detected, the types of various electric power towers can be automatically identified, and data information such as correct and reliable configuration, position and the like can be provided for inspection of the electric power towers.
A large number of experiments show that under the conditions of stable movement and little illumination change, the direct method is superior to the existing feature-based method in terms of precision and robustness, and the established three-dimensional point cloud image is obviously denser than the feature-point-based method, as shown in fig. 5 and 6.
The direct method is very different from other feature-based methods in two points:
feature-based methods calculate the position of the camera pose and map points by minimizing the re-projection error, while the direct law minimizes photometric errors (photometric error). By photometric error is meant that the minimized objective function is generally determined by the error between the images, rather than the geometric error after re-projection. Because of the limit of the number of the feature points, the information used by the feature-based method is far less than that of the direct method, and the positioning and mapping effects are limited.
The direct method puts data association and pose estimation (pose estimation) in a unified nonlinear optimization problem, while the feature-based method solves the problem step by step, namely, firstly, the association between the data is solved through matching feature points, and then the pose is estimated according to the association. The two steps are usually independent, and in the second step, outliers in the data association can be judged by the reprojection error and can also be used for correcting the matching result. When the movement is intense, the prior brought by the first step exists in the method based on the characteristics, so that the characteristic points are not easy to trace and fail, and the effect is better than that of the direct method.
Experiments show that by utilizing the advantages of the direct method and the characteristic-based method, the two methods are combined to obtain a more accurate and robust result, loop detection and repositioning are difficult to be carried out by the direct method, and characteristic points are required to be extracted to a certain extent to complete the loop detection and repositioning functions. Therefore, the invention combines the characteristic point method by calculating the score of the Shi-Tomas corner point and selecting the category of the point as the characteristic point or the edge point according to the threshold value at one time. The corner points of Shi-Tomas are characterized by insensitivity to variations in brightness and contrast and rotational invariance. The invention adopts a direct method at the stage of gentle pose change; the pose changes quickly, and a feature-based method is adopted. And simultaneously, in a dense map generated by a direct method, the colinear property and coplanarity of the features are supplemented.
The basic principle is that a target pixel point is taken as a center, the change condition of gray curvature in a window is calculated, and the maximum point of curvature change value is selected as a characteristic point. The calculation method is as follows:
set G x,y The gray level change amount when the neighborhood window W of the pixel point (x, y) moves (u, v) is obtained by performing diagonalization processing and quadratic calculation according to Talyor series expansion when the movement (u, v) is a local minimum:
wherein I is the image gray level, I x 、I y First partial derivatives in x and y directions respectively,covariance matrix for image gradient, +.>As a window function of window W, active noise reduction smoothing, lambda 1 、λ 2 And R is a two-dimensional rotation matrix, wherein the characteristic value of the matrix M is the characteristic value of the matrix M.
As shown in fig. 7 and 8, lambda 1 、λ 2 The eigenvalues indicate the distribution of the gray values of the pixels of the image. When lambda is 1 、λ 2 All smaller, the imageThe window does not have obvious gray level change when moving in any direction, which indicates that the corresponding pixel point in the image window is in a smooth area when lambda 1 、λ 2 When one of the two pixels is bigger and the other is smaller, the gray level change of the image window in one direction is more obvious, and the gray level change in the other direction is not obvious, which means that the corresponding pixel point in the image window is positioned at the boundary of the object, when lambda is 1 、λ 2 When the curvature or the gray gradient change of the pixel point in the image window moving along any direction is larger, the point is the required characteristic point.
When extracting the characteristic points of a whole image, the characteristic values of all pixel points in the image need to be calculated, and the calculated amount is large. Since the matrix M is a real symmetric matrix, set:
then:
where tr (M) is the trace of matrix M and det (M) is its determinant. To improve computational efficiency, λ is generally avoided by tr (M) and det (M) 1 、λ 2 Is obtained by the method. Thereby, corner feature detection functions are obtained:
R(x,y)＝det(M)-k(tr(M)) 2 ＝(AB-C 2 )-k(A+B) 2
here, k=0.04-0.06, which is an empirical constant, may be adaptively modified according to actual scenes.
Given a threshold T, for a pixel point (x, y), when R (x, y) is greater than T and the maximum value in the local neighborhood window, the pixel point (x, y) is the corner point.
In summary, a block diagram of the classical Harris algorithm can be obtained, as shown in fig. 9.
If the smaller of the two eigenvalues is larger than the minimum threshold value, a strong corner point is obtained. Namely:
R(x,y)＝min(λ 1 ,λ 2 )
since a larger uncertainty depends on smaller feature values, good feature points can be found by finding the maximum of the small feature values.
The Shi-Tomas algorithm gives better results than the Harris algorithm in most cases. The operator may be implemented using the goodfeaturestrack function in Opencv.
And finally, realizing three-dimensional reconstruction of the power line by combining with the airborne edge equipment through comparative research of an algorithm.
The invention considers the defects of the existing inspection mode and supplements and perfects the defects, and the invention has the following direct benefit and indirect benefit, wherein the invention comprises a model identification algorithm based on airborne edge equipment, a dynamic planning algorithm, a three-dimensional point cloud data mining method, an inspection mode based on GPS and the like.
(1) Direct benefit
At present, an inspection mode of RTK positioning is adopted, and the positioning accuracy of the RTK is depended, but real-time dynamic planning and safe route inspection are lacked. The invention can be used for carrying out safety inspection on the route and the waypoint in the existing autonomous inspection, and can ensure that the image of the shot target is more standard and has higher quality.
The invention provides a totally new autonomous inspection method in a GPS mode, solves the autonomous inspection problem of the RTK-free area, can reduce the labor amount of flying workers, and ensures that the inspection process is more intelligent, flexible and convenient.
(2) Indirect benefit
The invention researches an algorithm based on edge computing equipment, which can be popularized and applied to other related fields in the power grid, such as channel inspection, ground wire inspection and the like. Meanwhile, the achievement of the invention further comprises algorithms such as dynamic planning algorithms, three-dimensional point cloud data mining and the like, and the algorithms are greatly innovated in scientific research and application, so that not only can the foundation be laid for subsequent research, but also the algorithms can be popularized to other computing platforms such as mobile terminals, service terminals and the like.
The unmanned aerial vehicle line inspection based on autonomous learning of edge is mainly aimed at realizing unmanned aerial vehicle line inspection based on autonomous decision and edge calculation under the condition of no RTK, the inspection efficiency of the power transmission line is effectively improved, the autonomous inspection threshold is further reduced, and the unmanned aerial vehicle line inspection system has great value significance for the actual conditions of less inspection flying hands, limited inspection range and the like of the power transmission line of each unit at present, and has very wide application prospect.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the content and principle of the present invention should be included in the protection scope of the present invention.

Claims (6)

1. The monocular vision real-time three-dimensional reconstruction method based on the airborne edge equipment is characterized by comprising the following specific steps of:
when the digital modeling of the power line corridor is carried out, decomposing and assembling a design standardized model to decompose a pole tower into a tower crown, a tower foundation and an electric element assembly, and then modeling the assembly type of the power transmission line pole tower model;
analyzing the electrical characteristics, the dimensional configuration and the position information of each power tower component to establish an attribute set of the power tower component, and effectively and accurately describing and recording key characteristics of the power tower component;
in order to ensure seamless splicing among the tower crown, the tower foundation and the electrical components, association relations and constraint rules among different types of components in the model library also need to be defined in detail; the method comprises the steps of spatial position constraint, size constraint and functional constraint, and establishing an association relation model and a corresponding constraint rule of connection among all components of a corresponding power tower;
constructing a power pole tower component knowledge base; constructing an inference rule of the power pole tower component to detect whether the corresponding component body meets the rule requirements such as consistency, inclusion relation among classes and the like; the correctness of the electric power tower model constructed by the components can be detected, the types of various electric power towers can be automatically identified, and data information such as correct and reliable configuration, position and the like can be provided for inspection of the electric power towers;
a direct method is adopted at the stage of gentle pose change; the pose changes quickly, and a feature-based method is adopted; the class of the points is selected as the characteristic points or the edge points at one time according to the threshold value by calculating the points of the Shi-Tomas corner points, and the three-dimensional reconstruction of the power line is finally realized by combining with the airborne edge equipment through the comparative research of the algorithm.
2. The method for monocular vision real-time three-dimensional reconstruction based on airborne edge equipment of claim 1, wherein constructing a knowledge base of power tower components specifically comprises (a) analyzing important terms in the power tower components; (b) defining power tower component classes and class hierarchy; (c) defining attributes for each power tower component class; (d) defining facets of the power tower assembly attribute; (e) creating a corresponding power tower assembly instance.
3. The monocular vision real-time three-dimensional reconstruction method based on the airborne edge equipment according to claim 1, wherein the calculation method combining the direct method and the indirect method during three-dimensional reconstruction is as follows:
set G x,y The gray level change amount when the neighborhood window W of the pixel point (x, y) moves (u, v) is obtained by performing diagonalization processing and quadratic calculation according to Talyor series expansion when the movement (u, v) is a local minimum:
wherein I is the image gray level, I x 、I y First partial derivatives in x and y directions respectively,covariance matrix for image gradient, +.>As a window function of window W, active noise reduction smoothing, lambda 1 、λ 2 The characteristic value of the matrix M is represented by R, which is a two-dimensional rotation matrix;
λ 1 、λ 2 the characteristic value indicates the distribution condition of the gray value of the image pixel; when lambda is 1 、λ 2 When the image window is smaller, no obvious gray level change exists when the image window moves in any direction, which indicates that the corresponding pixel point in the image window is in a smooth area, when lambda 1 、λ 2 When one of the two pixels is bigger and the other is smaller, the gray level change of the image window in one direction is more obvious, and the gray level change in the other direction is not obvious, which means that the corresponding pixel point in the image window is positioned at the boundary of the object, when lambda is 1 、λ 2 When the curvature or the gray gradient change of the pixel point in the image window moving along any direction is larger, the point is the required characteristic point.
4. The monocular vision real-time three-dimensional reconstruction method based on the airborne edge equipment as claimed in claim 1, wherein when the feature point extraction is carried out on a whole image during three-dimensional reconstruction, the feature values of all pixel points in the image are required to be calculated, and the calculated amount is relatively large; since the matrix M is a real symmetric matrix, set:
then:
tr(M)＝λ 1 +λ 2 ＝A+B
det(M)＝λ 1 λ 2 ＝AB-C 2 )
where tr (M) is the trace of matrix M and det (M) is its determinant; to improve computational efficiency, λ is generally avoided by tr (M) and det (M) 1 、λ 2 Is calculated; thereby, corner feature detection functions are obtained:
R(x,y)＝det(M)-k(tr(M)) 2 ＝(AB-C 2 )-k(A+B) 2
here, k=0.04-0.06, which is an empirical constant, may be adaptively modified according to actual scenes.
5. The monocular vision real-time three-dimensional reconstruction method based on the airborne edge equipment according to claim 1, wherein a threshold value T is given in three-dimensional reconstruction, and when R (x, y) is larger than T and the maximum value in a local neighborhood window is larger than R (x, y), the pixel point (x, y) is the corner point.
6. The method for monocular vision real-time three-dimensional reconstruction based on airborne edge equipment according to claim 1, wherein if the smaller one of the two feature values is larger than the minimum threshold value, then a strong angular point is obtained; namely:
R(x,y)＝min(λ 1 ,λ 2 )
since a larger uncertainty depends on smaller feature values, good feature points can be found by finding the maximum of the small feature values.
CN202311564747.5A 2023-11-22 2023-11-22 Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment Pending CN117611740A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311564747.5A CN117611740A (en) 2023-11-22 2023-11-22 Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311564747.5A CN117611740A (en) 2023-11-22 2023-11-22 Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment

Publications (1)

Publication Number Publication Date
CN117611740A true CN117611740A (en) 2024-02-27

Family

ID=89958987

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311564747.5A Pending CN117611740A (en) 2023-11-22 2023-11-22 Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment

Country Status (1)

Country Link
CN (1) CN117611740A (en)

Similar Documents

Publication Publication Date Title
CN112258618B (en) Semantic mapping and positioning method based on fusion of prior laser point cloud and depth map
Yang et al. Visual perception enabled industry intelligence: state of the art, challenges and prospects
CA2994508C (en) Vegetation management for power line corridor monitoring using computer vision
US11816907B2 (en) Systems and methods for extracting information about objects from scene information
CA2994511C (en) Condition detection using image processing
CN103679674B (en) Method and system for splicing images of unmanned aircrafts in real time
CN110956651A (en) Terrain semantic perception method based on fusion of vision and vibrotactile sense
US20230298196A1 (en) Geospatial object geometry extraction from imagery
US11776104B2 (en) Roof condition assessment using machine learning
Ahmed et al. Inspection and identification of transmission line insulator breakdown based on deep learning using aerial images
Hu et al. Geometric feature enhanced line segment extraction from large-scale point clouds with hierarchical topological optimization
Yi et al. Detection and localization for lake floating objects based on CA-faster R-CNN
CN114037895A (en) Unmanned aerial vehicle pole tower inspection image identification method
Li et al. An efficient point cloud place recognition approach based on transformer in dynamic environment
Li et al. 3D-SeqMOS: A novel sequential 3D moving object segmentation in autonomous driving
Wang et al. Target detection for construction machinery based on deep learning and multisource data fusion
CN117611740A (en) Monocular vision real-time three-dimensional reconstruction method based on airborne edge equipment
Zhou et al. Three dimensional fully autonomous inspection method for wind power employing unmanned aerial vehicle based on 5G wireless communication and artificial intelligence
Ma et al. Robust visual-inertial odometry with point and line features for blade inspection UAV
Yildirim et al. Autonomous Ground Refuelling Approach for Civil Aircrafts using Computer Vision and Robotics
Yang et al. Locator slope calculation via deep representations based on monocular vision
Lawton et al. Knowledge based vision for terrestrial robots
Zhou et al. UAV based indoor localization and objection detection
Morando et al. Thermal and Visual Tracking of Photovoltaic Plants for Autonomous UAV Inspection. Drones 2022, 6, 347
Lu et al. MStrans: Multiscale Vision Transformer for Aerial Objects Detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination