CN115082254A - Lean control digital twin system of transformer substation - Google Patents
Lean control digital twin system of transformer substation Download PDFInfo
- Publication number
- CN115082254A CN115082254A CN202210254163.7A CN202210254163A CN115082254A CN 115082254 A CN115082254 A CN 115082254A CN 202210254163 A CN202210254163 A CN 202210254163A CN 115082254 A CN115082254 A CN 115082254A
- Authority
- CN
- China
- Prior art keywords
- power transmission
- point cloud
- transformer substation
- module
- transmission tower
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 claims abstract description 99
- 238000010276 construction Methods 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 34
- 238000005259 measurement Methods 0.000 claims abstract description 30
- 238000001514 detection method Methods 0.000 claims abstract description 17
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000012800 visualization Methods 0.000 claims abstract description 11
- 230000007246 mechanism Effects 0.000 claims abstract description 9
- 230000005477 standard model Effects 0.000 claims abstract description 9
- 238000013528 artificial neural network Methods 0.000 claims abstract description 8
- 238000007726 management method Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000005484 gravity Effects 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 5
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000002372 labelling Methods 0.000 abstract description 5
- 238000011160 research Methods 0.000 abstract description 4
- 230000000007 visual effect Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 10
- 239000011159 matrix material Substances 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000007935 neutral effect Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Entrepreneurship & Innovation (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Development Economics (AREA)
- Biophysics (AREA)
- Educational Administration (AREA)
- Mathematical Physics (AREA)
- Game Theory and Decision Science (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Operations Research (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Computer Graphics (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a lean control digital twinning system for a transformer substation, which comprises: the three-dimensional visualization module is used for acquiring three-dimensional information of buildings in the transformer substation and generating standard models of the buildings at different construction stages; acquiring a plurality of images of a building in the transformer station from different viewing angles in the construction process to obtain a reconstruction model of the building; the three-dimensional measurement module is used for acquiring original point clouds of the power transmission line and the power transmission tower, and calculating the sag angle of the power transmission line and the inclination angle of the power transmission tower after the point clouds of the power transmission line and the point clouds of the power transmission tower are separated independently; the anomaly detection module is used for inputting a power line image to be detected or a power equipment image to be detected as an original image into a depth neural network based on an attention mechanism to obtain a prediction boundary frame and a prediction type of a target in the original image; and the remote assistance module is used for obtaining the holographic labeling scene. The intelligent linkage of digital holographic research and judgment of the transformer substation construction is realized.
Description
Technical Field
The invention belongs to the technical field of transformer substation management and control, and particularly relates to a lean management and control digital twinning system for a transformer substation.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The electric energy is indispensable as the blood vessels for normal operation of the modern social life. The capital construction, the acceptance, the operation of transformer substation in the energy framework, maintenance are relatively independent, rely on each other again, and construction personnel need carry out the capital construction to the transformer substation according to the plane drawing, and managers need carry out the manual work to the capital construction result and accept, and operation and maintenance personnel need monitor data, and the personnel of patrolling and examining need overhaul equipment, and it is many to drag information, and the incidence is complicated, leads to transformer substation construction not directly perceived, not meticulous, the risk is big, inefficiency.
With the rapid development of computer technology, the digital twin becomes the core and key of the industrial internet landing. The digital twinning refers to integrating a multidisciplinary and multiscale simulation process by fully utilizing data such as a physical model, sensor data and operation history. Computer virtual model information can be utilized to reflect the full life cycle process of real physical devices through three-dimensional visualization. Simply speaking, the equivalent mapping of the information world to the physical world makes the change process which cannot be observed and is difficult to observe possible. In order to realize the digitization and virtualization of all elements of the transformer substation, the real-time and visualization of all states, and the synergy and intellectualization of management decisions, it is necessary to integrate the digital twin technology into the construction process of the transformer substation. However, the prior art cannot realize accurate sag measurement of a power transmission line, inclination measurement of a power transmission tower, abnormal detection of electrical equipment and the like, and cannot realize digital holographic research and judgment intelligent linkage for substation construction.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a lean control digital twin system for a transformer substation, solves the problems of unobtrusiveness, inexperience, high cost and high risk in the aspects of construction, acceptance, operation and management of the transformer substation, and realizes the intelligent linkage of digital holographic research and judgment on the construction of the transformer substation.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention discloses a lean control digital twinning system for a transformer substation, which comprises:
the three-dimensional visualization module is used for acquiring three-dimensional information of buildings in the transformer substation and generating standard models of the buildings at different construction stages; acquiring a plurality of images of the building in the transformer substation from different visual angles in the construction process to obtain a reconstruction model of the building, comparing the reconstruction model with the standard model in the same construction stage, and judging whether the progress of the planning is met;
the three-dimensional measurement module is used for acquiring original point clouds of the power transmission line and the power transmission tower, and calculating sag angles of the power transmission line and inclination angles of the power transmission tower after the point clouds of the power transmission line and the point clouds of the power transmission tower are separated independently;
and the abnormity detection module is used for inputting the power line image to be detected or the power equipment image to be detected as an original image into the depth neural network based on the attention mechanism to obtain a predicted boundary frame and a predicted type of the target in the original image.
And further, the system also comprises a remote assistance module which is used for sending the video to be annotated to the instructor, receiving the marking data returned by the instructor, and rendering the marking data to the video to be annotated to obtain the holographic annotation scene.
Further, the three-dimensional visualization module comprises a construction planning module and a reconstruction module;
the construction planning module is used for reading three-dimensional information of buildings in the transformer substation, and adding time information on the basis of the three-dimensional information to form four-dimensional construction planning information;
the reconstruction module is used for acquiring a plurality of images of a building in the transformer station in the construction process from different viewing angles, obtaining a reconstruction model of the building through a multi-view geometric three-dimensional reconstruction algorithm, comparing the reconstruction model with the standard model in the same construction stage, and judging whether the construction progress is met.
Further, the multi-view geometric three-dimensional reconstruction algorithm specifically comprises the following steps:
performing sparse point cloud reconstruction based on a plurality of images of the building with different viewing angles to obtain sparse point cloud;
estimating a depth map based on the sparse point cloud, and projecting depth pixels to a three-dimensional space according to the depth of each pixel in the depth map to obtain the dense point cloud;
and performing surface reconstruction based on the dense point cloud to obtain a reconstruction model of the building.
Further, the depth neural network based on the attention mechanism extracts features of the original image by using a backbone network, performs data fusion on the extracted features to obtain fusion features, and obtains a predicted boundary frame and a predicted category of a target in the original image based on the fusion features.
Further, the three-dimensional measurement module includes:
the point cloud acquisition module is used for acquiring original point clouds of a power transmission line and a power transmission tower in a transformer substation;
the interactive measurement module is used for removing pure ground object point clouds in the original point clouds to obtain target point clouds; dividing the target point cloud into a plurality of subspaces, calculating the density of each subspace, and judging whether each subspace belongs to the point cloud of the power transmission line or the point cloud containing trees and the power transmission tower based on the density; screening and separating the point cloud of the power transmission tower based on the point cloud containing the trees and the power transmission tower;
the sag measuring module is used for calculating sag of the power transmission line based on the power transmission line point cloud;
and the inclination measuring module is used for calculating the inclination angle of the power transmission tower based on the power transmission tower point cloud.
Further, the method for screening and separating the point cloud of the power transmission tower is an Euclidean clustering method.
Further, the specific method for eliminating the pure ground object point cloud in the original point cloud comprises the following steps: dividing the original point cloud into a plurality of subspaces, calculating the point cloud elevation difference of each subspace, judging whether each subspace is a pure ground object point cloud or not based on the point cloud elevation difference, and after the pure ground object point cloud is eliminated from the original point cloud, performing subspace division again, and judging and eliminating the pure ground object point cloud.
Further, the calculation method of the inclination angle of the transmission tower comprises the following steps: and (3) optionally selecting two planes parallel to the ground plane in the power transmission tower point cloud, respectively calculating the gravity center points of the two planes, and connecting the two gravity center points into a straight line, wherein the included angle between the straight line and the ground plane is the inclination angle of the power transmission tower.
Further, the calculation method of the sag of the power transmission line comprises the following steps: respectively determining the highest points on two sides of the middle point of the power transmission line, connecting the two highest points to obtain a line segment, determining the intersection point from the lowest point of the power transmission line to the vertical line of the line segment, wherein the distance between the intersection point and the lowest point is the sag of the power transmission line.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a lean management and control digital twinning system for a transformer substation, which not only realizes real-time and accurate monitoring of sag of a power transmission line and inclination angle of a power transmission tower of the transformer substation, but also realizes three-dimensional reconstruction of a construction building, target prediction and holographic labeling of a power transmission line image or an electrical equipment image and the like, solves the problems of incompleteness, high cost and high risk in the aspects of construction, acceptance, operation and management of the transformer substation, and realizes intelligent linkage of digital holographic research and judgment on construction of the transformer substation.
The invention provides a lean control digital twin system for a transformer substation, which realizes accurate measurement of sag of a power transmission line and inclination angle of the power transmission tower by judging and separating point cloud data of trees, the power transmission tower and the power transmission line.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is an overall functional diagram of a lean control digital twin system of a substation of the present invention;
FIG. 2 is a cloud platform of a Shandong map with a streamer effect according to the present invention;
FIG. 3 is a schematic diagram of a substation digital twinning system primary interface of the present invention;
FIG. 4 is a schematic view of a construction planning interface of the present invention;
FIG. 5 is a flow chart of a multi-view geometric three-dimensional reconstruction algorithm of the present invention;
FIG. 6 is a schematic diagram of the three-dimensional reconstruction effect of the present invention;
FIG. 7 is a schematic diagram of the raw point cloud data of the present invention;
fig. 8 is a schematic view of a transmission line sag measurement of the present invention;
fig. 9 is a schematic diagram of the transmission tower inclination measurement of the present invention;
FIG. 10 is a schematic diagram of the defect detection step of the present invention;
FIG. 11 is a schematic view of object recognition in accordance with the present invention;
FIG. 12 is a schematic view of defect detection of the present invention;
FIG. 13 is a first person perspective view of the present invention;
FIG. 14 is a schematic view of the unmanned aerial vehicle inspection animation of the present invention;
FIG. 15 is a flow diagram of the remote assisted virtual annotation of the present invention;
FIG. 16 is a schematic illustration of a 2D line marking timing embodiment of the present invention;
FIG. 17 is a schematic diagram of a 3D object picture mapped after the annotation is completed according to the present invention.
Detailed Description
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Referring to fig. 1, the present embodiment discloses a lean control digital twin system of a transformer substation, which includes: construction planning and three-dimensional visual module of work progress, based on the three-dimensional measurement module of multisource heterogeneous data, based on the unusual detection module of artificial intelligence's power equipment and based on mixed reality's remote assistance module are used for solving the transformer substation construction respectively, accept, operate (on duty) and manage (patrol and examine) the aspect exist not directly observe, not meticulous, with high costs, the big problem of risk, still include map display module, specific:
the cloud platform display module is used for acquiring a global map cloud platform display instruction and returning to a global map cloud platform with a streamer effect; the method comprises the steps of obtaining a local map cloud platform display instruction, returning to a local map cloud platform with a streamer effect, enabling a corresponding map to float up and down, namely clicking a certain province, enabling the map of the certain province to float up and down, for example, clicking Shandong province, enabling the map of the Shandong province to float up and down, and showing a selected visual effect. And entering a corresponding local map cloud platform for next operation (wherein, the three methods are mainly used as Transform, Vector3 and Material methods, all of which are the classes in C #, the Transform is used for changing the coordinate, the rotation angle and the zoom scale of the map, the Vector3 is used for realizing the change of the three-dimensional coordinate of the map, and the Material is used for realizing the change of the map Material, such as changing from a white map to a black map).
Specifically, the global map may be a national map. The local map may be a map of provinces or downtown areas, as shown in fig. 2.
The mode of returning to the global map or the local map cloud platform is as follows: and modeling a global map or a local map by using 3DSMAX software, rendering by using a Shader of a Unity engine after importing the global map or the local map into Unity, and displaying.
And the map display module is also used for acquiring a transformer substation display instruction and returning the transformer substation digital twin system main interface.
Taking the yuhuang temple in the Shandong Shannan Shanxi Yuhuang prefecture as an example, the user clicks the Shandong Yunan temple, as shown in FIG. 2, to obtain a transformer substation display instruction of the Shandong Jinan Yuhuang temple, and enters a main interface of a digital twin system of the transformer substation of the Shandong Jinan Yuhuang temple, as shown in FIG. 3.
And after entering a main interface of the digital twin system of the transformer substation, responding to a starting instruction of the three-dimensional visualization module, the three-dimensional measurement module, the abnormity detection module or the remote assistance module, and starting the three-dimensional visualization module, the three-dimensional measurement module, the abnormity detection module or the remote assistance module.
The three-dimensional visualization module for construction planning and construction process comprises a construction planning module for planning before construction and a reconstruction module for supervision of construction progress:
(1) the construction planning module takes Qt as an application framework and OpenGL technology as a core, designs a visual interactive interface, as shown in fig. 4, and is configured to read three-dimensional information (three-dimensional model information, position planning information, and assembly process information) of a building in a substation, add time information on the basis of the three-dimensional information to form visual four-dimensional construction planning information (4D-BIM information, which is standard models of buildings at different construction stages), and perform planning of a single object or perform overall planning, thereby adapting to requirements of a construction planning process.
(2) The reconstruction module is used for reconstructing a building, namely, a multi-view geometric three-dimensional reconstruction method is adopted to reconstruct a house with block characteristics, and the multi-view geometric three-dimensional reconstruction method is compared with the standard model in the same construction stage to judge whether the building conforms to the planning progress.
The multi-view geometric reconstruction algorithm process is as shown in fig. 5, and firstly, videos of different angles shot by an unmanned aerial vehicle around a building need to be acquired, then the videos are scattered into a plurality of images, and the images are used as input of a multi-view geometric three-dimensional reconstruction algorithm. The multi-view geometric three-dimensional reconstruction algorithm comprises the following specific steps:
based on a plurality of images of the building with different visual angles, SFM sparse point cloud reconstruction is carried out to obtain sparse point cloud: (a) acquiring a plurality of RGB images of a building at different viewing angles; (b) considering the problems of size and rotation, selecting an SIFT operator to extract the features of each image; (c) carrying out feature matching between adjacent frame images and establishing track (tracking) between the images, and carrying out pairwise matching on each pair of images through Euclidean distance; (d) for each image matching pair, calculating a basic matrix (representing a mapping relation from a point on one image to a point on the other image) of the camera, solving an essential matrix of the camera through the basic matrix (the basic matrix of a normalized plane is an essential matrix), then carrying out SVD (singular value decomposition) on the essential matrix to obtain a rotation matrix R and a translation matrix T, and correcting the distortion of the camera (the distortion correction can directly call an API (application program interface) packaged in Opencv), namely correcting the image; (e) and performing trigonometric calculation according to R, T and the corrected image point coordinates to obtain three-dimensional points, namely sparse point clouds.
Estimating a depth map based on the sparse point cloud, projecting depth pixels to a three-dimensional space according to the depth of each pixel in the depth map, and obtaining the dense point cloud: firstly, selecting seed points, screening out 3D points which can be projected on a reference image from sparse point cloud, and projecting the points on at least 1 frame of other images; after the 3D point is projected on the reference image, taking the distance from the 3D point to the origin of a camera coordinate system of the reference image as the initial depth of a pixel of the projection point, and taking the direction from the 3D point to the origin of the camera coordinate system as an initial normal vector; then, the depth and normal vector of the seed point are optimized, specifically, a square template is established by taking a projection pixel point as a center, all pixels in each template are projected into a space 3D coordinate, one template is taken as an example, the 3D point in one template is projected onto images of other visual angles to obtain pixel points of the projection point, and the depth and normal vector information of the pixel points are optimized to enable the projection difference between the template on a reference image and the template on other views to be as small as possible; and finally, according to the depth of each pixel in the depth map, projecting the depth pixel points to a three-dimensional space by using an inverse projection matrix of the camera to obtain dense point cloud.
And thirdly, performing surface reconstruction based on the dense point cloud to obtain a reconstruction model of the building: and selecting Delaunay triangularization surface reconstruction, namely tetrahedron expression based on dense point cloud. The method comprises the steps of triangularizing dense point cloud in Delaunay, converting a spatial tetrahedron set into a Graph structure, enabling each tetrahedron to correspond to one node of a Graph, connecting adjacent nodes by edges, calculating a data item of each node by using prior visibility information, further performing secondary classification on the Graph nodes by using a Graph cuts algorithm, enabling space boundaries after the secondary classification to be reconstructed surfaces, enabling reconstruction effects to be as shown in figure 6, introducing a reconstruction model of a building into a construction planning function, and comparing the reconstruction model with a standard model under a planning date in terms of form and volume so as to judge whether the planned construction progress is met or not and achieve progress control.
The three-dimensional measurement module based on multi-source heterogeneous data is mainly used for acquiring point cloud data of a power transmission line and a power transmission tower, and separating the point cloud of the power transmission line and the point cloud of the power transmission tower separately to perform sag measurement of the power transmission line and inclination measurement of the power transmission tower. The three-dimensional measurement module based on the multi-source heterogeneous data comprises a point cloud acquisition module and an automatic measurement module, wherein the automatic measurement module comprises a sag measurement module and an inclination measurement module. In particular, the method comprises the following steps of,
the point cloud acquisition module is used for acquiring point cloud data, namely original point cloud I, of a power transmission line and a power transmission tower, which are acquired by laser radar equipment carried by a unmanned plane in Xinjiang, and is shown in FIG. 7;
for sag measurement and tilt measurement, the power transmission line point cloud and the power transmission tower point cloud need to be separated separately. An automatic measurement module configured to: (a) firstly, solving a boundary value of an original point cloud I, and selecting a proper step length dx to divide the original point cloud I into n subspaces along an X axis; (b) then, counting the maximum value and the minimum value of each subspace point cloud in the Z-axis direction, and recording the difference between the maximum value and the minimum value as the point cloud elevation difference Dz of the subspace; (c) the average of all the subspace height differences needs to be calculated next
Will be provided with
As a threshold value; if it is
Dividing the subspace into a non-pure ground object point cloud; if it is
Dividing the subspace into pure ground object point clouds which need to be removed; (d) marking the new point cloud without the pure ground object point cloud as I 1 Re-executing steps (a) - (c) to obtain I 1 The data points in the method are divided into two types of ground object points and non-ground object points, the ground object point cloud is removed, and a new point cloud I is obtained 2 To this I 2 The system comprises a power transmission tower point cloud, a power transmission line point cloud and a small amount of trees or other high-altitude object point clouds. In order to extract the power transmission tower point cloud, other point clouds need to be rejected. From the density distribution, the point cloud of the transmission tower and a small number of trees orThe projection density of other high-altitude objects is greater than that of the power line points. Therefore, the power transmission tower point clouds are separated by density differences, and similarly, I 2 Dividing into m × n subspaces, and calculating the density ρ of each subspace m,n Then the density average is obtained
It is used as another threshold when the density of each subspace is less than the density average, i.e.
A power transmission line point cloud is obtained; when the density of each subspace is greater than the density average, i.e.
The point cloud of the power transmission tower and a small number of trees or other high-altitude objects is marked as I 3 . At this point, the point cloud of the transmission line is separated and recorded as I L 。
An automatic measurement module further configured to: for I 3 Analyzing the characteristics that the power transmission tower has a uniform size, and the point cloud of the power transmission tower can be screened and separated through size comparison, wherein an Euclidean clustering method is adopted, and the method specifically comprises the following steps: for point cloud I 3 At any point P (x) in 0 ,y 0 ,z 0 ) Using Kd-Tree neighbor search algorithm to obtain k points nearest to P, and calculating Euclidean distance D between the k points and P o As shown in equation (1):
wherein (x) i ,y i ,z i ) The three-dimensional coordinates of the ith point.
(a) Selecting proper threshold value through experiments
Will be provided with
Are clustered in the set Q. Then, selecting points except P in the set Q, and clustering again until the number of elements in Q is not increased any more, and finishing a clustering process; (b) replacing points except the set Q as P ', and recreating the set Q'; repeating the processes (a) - (b) until the cloud space I is reached 3 All points in the middle are clustered into some set. The point cloud at the moment comprises a small number of trees and the point cloud of the power transmission tower, the length-width ratio is used as a characteristic value of each cluster set according to the fact that the cluster sets of the power transmission tower have the same length-width ratio, a proper division scale is selected, the cluster sets which do not accord with the division scale are removed, the point cloud of the power transmission tower is the rest point cloud, and the point cloud is marked as I T 。
A sag measurement module configured to: as shown in fig. 8, for transmission line I L Measuring sag, connecting a left highest point A and a right highest point B of a midpoint of each section of the transmission line, recording a line segment as L, namely respectively determining the highest points on two sides of the midpoint of the transmission line, and connecting the two highest points to obtain a line segment; and (5) making a vertical line upwards from the lowest point C, crossing the L at a point D, wherein the distance between the CD and the L is the sag of the power transmission line.
A tilt measurement module configured to: as shown in fig. 9, for the transmission tower tilt measurement, two planes parallel to the XOY plane (ground plane) in the same transmission tower point cloud are arbitrarily selected, and the gravity center points of the two planes are respectively calculated and recorded as O 1 (x 1 ,y 1 ,z 1 ) And O 2 (x 2 ,y 2 ,z 2 ) And (3) connecting the two gravity centers into a straight line, wherein the included angle between the straight line and the ground plane is the inclination angle of the power transmission tower, as shown in formula (2).
The power equipment abnormity detection module based on artificial intelligence mainly comprises target identification which may harm the safety of a power transmission line and defect detection of power equipment, and the key technologies used are a deep neural network and an attention mechanism. Specifically, the method comprises the following steps: the unmanned aerial vehicle inspection module is used for acquiring an image of a power transmission line to be detected or an image of power equipment to be detected; and the abnormity detection module comprises a power transmission line target identification module and an electric power equipment defect detection module, and is used for acquiring a power transmission line image to be detected and an electric power equipment image to be detected as original images and inputting the original images into the depth neural network based on the attention mechanism. Specifically, an original image is input into a backhaul network, and feature extraction is performed; inputting the extracted features into a neutral network, and performing data fusion to obtain fusion features; and inputting the fusion characteristics into a Head network to obtain a prediction boundary frame and a prediction category of the target in the original image.
The training step of the deep neural network based on the attention mechanism, as shown in fig. 10, includes: the method comprises the steps of firstly performing Mosaic data enhancement before a data set enters a backbone network, randomly using 4 images in the data set, randomly zooming, and then randomly distributing for splicing, so that the detection data set is greatly enriched, and particularly, many small targets are added by random zooming, so that the robustness of the whole network is better. The network designs a self-adaptive Anchor calculation, and anchor frames with the length and the width set initially exist for different data sets. After the data processing is finished, entering a Backbone network (Backbone network) to perform feature extraction on the data, specifically: entering a Focus module to reduce information loss; then, 3 CSP modules with residual error structures are entered to play a role of down sampling, and the accuracy is ensured while the calculated amount is reduced; then, introducing an attention mechanism, firstly performing an Squeeze operation on the output of each CSP module, as shown in formula (3); then performing an Excitation operation as shown in formula (4), and finally performing a feature scaling operation as shown in formula (5),
wherein u is c Is the input tensor, H, W are the height and width of the input tensor, respectively.
s＝F ex (z,W)＝σ(g(z,W))＝σ(W 2 δ(W 1 z)) (4)
Wherein, W 1 And W 2 Are respectively fully connectedAnd (4) the weight of the next layer, delta is a Relu activation function, and sigma is a sigmoid activation function.
Wherein S is c Is the resulting attention matrix.
After the feature extraction is finished, entering a neutral network for data fusion to obtain fusion features; the combined structure of the FPN + PAN is designed by the Neck network, the FPN layer conveys strong semantic features from top to bottom, the PAN conveys strong positioning features from bottom to top, and feature aggregation is carried out on different detection layers from different trunk layers by pairwise connection.
Finally, the fusion features enter a Head network, vectors are output, a prediction boundary frame and a category of the target in the original image are generated and marked, and the effect is shown in fig. 11 and 12.
The power equipment abnormity detection module based on artificial intelligence further comprises a first person visual angle module, in order to enable workers to know the change of the operation cycle of the transformer substation in a better mode, the design integrates the virtual reality technology, the first person visual angle module is designed, and the workers can be physically present at the site of the transformer substation without going out, as shown in fig. 13.
In the power equipment abnormity detection module based on artificial intelligence, animation demonstration of unmanned aerial vehicle inspection is added, a leading-edge technology which cannot be observed or is difficult to observe is displayed in front of eyes, and the working personnel can be visually helped to understand the fusion process of the unmanned aerial vehicle, the 5G technology, the artificial intelligence technology and the digital visualization technology, as shown in fig. 14.
The mixed reality-based remote assistance module is based on a mixed reality technology, holographic virtual labeling is carried out by using HoloLens2 intelligent glasses as carriers, the main idea is that when a worker (a requester) encounters a difficult problem, a assistance request is sent to a remote expert (a director), request information is transmitted to the director through a server, and the director agrees to send a labeling request to the requester. Meanwhile, the video quality of the current requesting party needs to be detected, if the video quality is low, a result is sent to the remote assistance end, at the moment, a prompt that the current visual angle is improper is displayed on the assistance end equipment, and the mobile equipment can be labeled only by adjusting the distance properly by a worker; if the quality of the video to be marked is high, the sending result can be marked, the instructor marks the video and then sends the marked data to the requester through the server, the requester renders the marked data to the video to be marked to obtain a holographic marking scene, field workers can see the holographic marking scene through HoloLens2 intelligent glasses, the space communication limitation is broken, the remote instruction quality and efficiency are greatly improved, and the process is shown in FIG. 15. The pictures of the two parties during 2D annotation of the guiding party are shown in fig. 16, wherein the left picture is a HoloLens view angle, and the right picture is a picture during 2D line drawing annotation of the guiding party; after the labeling is finished, the three-dimensional label mapped under the HoloLens glasses is shown in FIG. 17, wherein the left image is the HoloLens visual angle, and the right image is the 3D real object picture mapped after the labeling is finished.
And integrating all the functional modules in the Unity engine.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.
Claims (10)
1. A transformer substation lean management and control digital twin system is characterized by comprising:
the three-dimensional visualization module is used for acquiring three-dimensional information of buildings in the transformer substation and generating standard models of the buildings at different construction stages; acquiring a plurality of images of a building in the transformer station at different viewing angles in the construction process to obtain a reconstruction model of the building, comparing the reconstruction model with the standard model at the same construction stage, and judging whether the construction progress is met;
the three-dimensional measurement module is used for acquiring original point clouds of the power transmission line and the power transmission tower, and calculating the sag angle of the power transmission line and the inclination angle of the power transmission tower after the point clouds of the power transmission line and the point clouds of the power transmission tower are separated independently;
and the abnormity detection module is used for inputting the power line image to be detected or the power equipment image to be detected as an original image into the depth neural network based on the attention mechanism to obtain a predicted boundary frame and a predicted type of the target in the original image.
2. The transformer substation lean management and control digital twin system as claimed in claim 1, further comprising a remote assistance module, configured to send a video to be annotated to a director, and after receiving marking data returned by the director, render the marking data to the video to be annotated to obtain a holographic annotation scene.
3. The transformer substation lean management and control digital twin system according to claim 1, wherein the three-dimensional visualization module comprises a construction planning module and a reconstruction module;
the construction planning module is used for reading three-dimensional information of buildings in the transformer substation, and adding time information on the basis of the three-dimensional information to form four-dimensional construction planning information;
the reconstruction module is used for acquiring a plurality of images of a building in the transformer station from different viewing angles in the construction process, and obtaining a reconstruction model of the building through a multi-view geometric three-dimensional reconstruction algorithm.
4. The transformer substation lean management and control digital twin system according to claim 3, wherein the multi-view geometric three-dimensional reconstruction algorithm comprises the following specific steps:
performing sparse point cloud reconstruction based on a plurality of images of the building with different viewing angles to obtain sparse point cloud;
estimating a depth map based on the sparse point cloud, and projecting depth pixels to a three-dimensional space according to the depth of each pixel in the depth map to obtain the dense point cloud;
and performing surface reconstruction based on the dense point cloud to obtain a reconstruction model of the building.
5. The transformer substation lean management and control digital twin system according to claim 1, wherein the attention mechanism-based deep neural network extracts features of the original image by using a backbone network, performs data fusion on the extracted features to obtain fusion features, and obtains a predicted boundary frame and a category of an object in the original image based on the fusion features.
6. The substation lean management and control digital twin system according to claim 1, wherein the three-dimensional measurement module comprises:
the point cloud acquisition module is used for acquiring original point clouds of a power transmission line and a power transmission tower in a transformer substation;
the interactive measurement module is used for removing pure ground object point clouds in the original point clouds to obtain target point clouds; dividing the target point cloud into a plurality of subspaces, calculating the density of each subspace, and judging whether each subspace belongs to the point cloud of the power transmission line or the point cloud containing trees and the power transmission tower based on the density; screening and separating the point cloud of the power transmission tower based on the point cloud containing the trees and the power transmission tower;
the sag measuring module is used for calculating sag of the power transmission line based on the power transmission line point cloud;
and the inclination measuring module is used for calculating the inclination angle of the power transmission tower based on the power transmission tower point cloud.
7. The transformer substation lean management and control digital twin system according to claim 6, wherein the method for screening and separating the point cloud of the power transmission tower is a Euclidean clustering method.
8. The transformer substation lean management and control digital twin system according to claim 6, wherein the specific method for eliminating the pure ground object point cloud in the original point cloud is as follows: dividing the original point cloud into a plurality of subspaces, calculating the point cloud elevation difference of each subspace, judging whether each subspace is a pure ground object point cloud or not based on the point cloud elevation difference, and after the pure ground object point cloud is eliminated from the original point cloud, performing subspace division again, and judging and eliminating the pure ground object point cloud.
9. The transformer substation lean management and control digital twin system according to claim 6, wherein the calculation method of the inclination angle of the transmission tower comprises the following steps: and (3) optionally selecting two planes parallel to the ground plane in the power transmission tower point cloud, respectively calculating the gravity center points of the two planes, and connecting the two gravity center points into a straight line, wherein the included angle between the straight line and the ground plane is the inclination angle of the power transmission tower.
10. The transformer substation lean management and control digital twin system according to claim 6, wherein the sag calculation method of the transmission line is as follows: respectively determining the highest points on two sides of the middle point of the power transmission line, connecting the two highest points to obtain a line segment, determining the intersection point from the lowest point of the power transmission line to the vertical line of the line segment, wherein the distance between the intersection point and the lowest point is the sag of the power transmission line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210254163.7A CN115082254A (en) | 2022-03-15 | 2022-03-15 | Lean control digital twin system of transformer substation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210254163.7A CN115082254A (en) | 2022-03-15 | 2022-03-15 | Lean control digital twin system of transformer substation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115082254A true CN115082254A (en) | 2022-09-20 |
Family
ID=83247795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210254163.7A Pending CN115082254A (en) | 2022-03-15 | 2022-03-15 | Lean control digital twin system of transformer substation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115082254A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115549078A (en) * | 2022-10-12 | 2022-12-30 | 国网山西省电力公司 | Power grid integration planning method based on digital twinning |
| CN116012626A (en) * | 2023-03-21 | 2023-04-25 | 腾讯科技（深圳）有限公司 | Material matching method, device, equipment and storage medium for building elevation image |
| CN116109684A (en) * | 2023-04-07 | 2023-05-12 | 国网智能电网研究院有限公司 | Online video monitoring two-dimensional and three-dimensional data mapping method and device for variable electric field station |
| CN117829685A (en) * | 2024-03-06 | 2024-04-05 | 中国水利水电第七工程局有限公司 | Engineering management system and method based on digital twin and mixed reality technology |
| CN118537929A (en) * | 2024-07-25 | 2024-08-23 | 浙江大华技术股份有限公司 | Object behavior analysis method, device and storage medium |
-
2022
- 2022-03-15 CN CN202210254163.7A patent/CN115082254A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115549078A (en) * | 2022-10-12 | 2022-12-30 | 国网山西省电力公司 | Power grid integration planning method based on digital twinning |
| CN115549078B (en) * | 2022-10-12 | 2023-08-11 | 国网山西省电力公司 | Power grid integration planning method based on digital twin |
| CN116012626A (en) * | 2023-03-21 | 2023-04-25 | 腾讯科技（深圳）有限公司 | Material matching method, device, equipment and storage medium for building elevation image |
| CN116109684A (en) * | 2023-04-07 | 2023-05-12 | 国网智能电网研究院有限公司 | Online video monitoring two-dimensional and three-dimensional data mapping method and device for variable electric field station |
| CN117829685A (en) * | 2024-03-06 | 2024-04-05 | 中国水利水电第七工程局有限公司 | Engineering management system and method based on digital twin and mixed reality technology |
| CN118537929A (en) * | 2024-07-25 | 2024-08-23 | 浙江大华技术股份有限公司 | Object behavior analysis method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115082254A (en) | Lean control digital twin system of transformer substation | |
| WO2022036867A1 (en) | Method for reconstructing three-dimensional model of transformer substation | |
| CN111753698B (en) | Multi-mode three-dimensional point cloud segmentation system and method | |
| CN112990310A (en) | Artificial intelligence system and method for serving electric power robot | |
| WO2023138062A1 (en) | Image processing method and apparatus | |
| CN110765528A (en) | Three-dimensional reconstruction transformer substation implementation method based on virtual simulation technology | |
| CN114758337B (en) | Semantic instance reconstruction method, device, equipment and medium | |
| KR20210115245A (en) | Intelligent dam management system based on digital twin | |
| CN110660125B (en) | Three-dimensional modeling device for power distribution network system | |
| CN114219855A (en) | Point cloud normal vector estimation method and device, computer equipment and storage medium | |
| CN111354028B (en) | Binocular vision-based power transmission channel hidden danger identification and tracking method | |
| CN110458142A (en) | A kind of face identification method and system merging 2D and 3D | |
| CN112084916A (en) | Automatic generation and diagnosis method for urban three-dimensional skyline contour line based on shielding rate | |
| CN115937461B (en) | Multi-source fusion model construction and texture generation method, device, medium and equipment | |
| CN114741768A (en) | Three-dimensional modeling method for intelligent substation | |
| CN115222884A (en) | Space object analysis and modeling optimization method based on artificial intelligence | |
| CN116030208A (en) | Method and system for building scene of virtual simulation power transmission line of real unmanned aerial vehicle | |
| CN117197388A (en) | Live-action three-dimensional virtual reality scene construction method and system based on generation of antagonistic neural network and oblique photography | |
| CN115564031A (en) | Detection network for glass defect detection | |
| CN113920254B (en) | Monocular RGB (Red Green blue) -based indoor three-dimensional reconstruction method and system thereof | |
| CN116977586A (en) | Live-action three-dimensional modeling method based on power grid GIS platform | |
| CN114782357A (en) | Self-adaptive segmentation system and method for transformer substation scene | |
| CN118247429A (en) | Air-ground cooperative rapid three-dimensional modeling method and system | |
| CN116152442B (en) | Three-dimensional point cloud model generation method and device | |
| CN117671130A (en) | Digital twin intelligent fishing port construction and use method based on oblique photography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |