CN114355977A - Tower type photo-thermal power station mirror field inspection method and device based on multi-rotor unmanned aerial vehicle - Google Patents
Tower type photo-thermal power station mirror field inspection method and device based on multi-rotor unmanned aerial vehicle Download PDFInfo
- Publication number
- CN114355977A CN114355977A CN202210003230.8A CN202210003230A CN114355977A CN 114355977 A CN114355977 A CN 114355977A CN 202210003230 A CN202210003230 A CN 202210003230A CN 114355977 A CN114355977 A CN 114355977A
- Authority
- CN
- China
- Prior art keywords
- image
- heliostat
- aerial vehicle
- unmanned aerial
- rotor unmanned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 86
- 238000007689 inspection Methods 0.000 title claims abstract description 22
- 238000009434 installation Methods 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 230000006870 function Effects 0.000 claims description 32
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 12
- 238000006073 displacement reaction Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 14
- 238000010248 power generation Methods 0.000 description 15
- 230000007797 corrosion Effects 0.000 description 4
- 238000005260 corrosion Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 239000006096 absorbing agent Substances 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 230000035772 mutation Effects 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 238000005338 heat storage Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a tower type photo-thermal power station mirror field inspection method and device based on a multi-rotor unmanned aerial vehicle, and the method comprises the following steps: carrying out internal reference calibration on the multi-rotor unmanned aerial vehicle lens by using a Zhang chessboard method; the method comprises the steps that a multi-rotor unmanned aerial vehicle is used for cruising and shooting above a mirror field of the tower type photo-thermal power station to obtain an image sequence of the mirror field; preprocessing an image sequence to remove interference factors in the image; for each preprocessed image, extracting the feature points of the heliostat in each image, and performing feature matching in different image sequences with the same mirror to realize three-dimensional reconstruction to obtain a three-dimensional model of the heliostat in a mirror field; through the heliostat three-dimensional model, the included angle between the heliostat and the installation horizontal plane in the mirror field is calculated, and the multi-rotor unmanned aerial vehicle can patrol the position and pose of the heliostat in the mirror field. The method and the device have the advantages that the detection cost of the heliostat of the photo-thermal power station is reduced, the detection efficiency is improved, and the detection precision of the angle and the like of the heliostat is higher.
Description
Technical Field
The invention relates to the technical field of three-dimensional modeling of image sequences, in particular to a method for realizing inspection of a mirror field of a photo-thermal power station by acquiring an image sequence for three-dimensional modeling based on multi-rotor unmanned aerial vehicle photography.
Background
With the rapid development of economy and the increasing consumption of energy by people, the excessive development and utilization of fossil energy causes environmental pollution and energy crisis. Solar power generation is an important renewable green resource, has the advantages of rich resources, no pollution, renewability and the like, and is increasingly valued by people in development and utilization.
At present, the mature solar power generation technology is solar photovoltaic power generation and solar photo-thermal power generation. The solar photo-thermal power generation technology is divided into tower type solar photo-thermal power generation, groove type solar photo-thermal power generation and disc type solar photo-thermal power generation. The tower type solar thermal power generation system mainly comprises a light-gathering and heat-collecting subsystem, a heat storage subsystem and a power subsystem. The light and heat collecting subsystem is divided into a light condensing subsystem and a heat collecting subsystem, wherein the light condensing subsystem mainly comprises a heliostat, and the heliostat is a light condensing device which comprises a reflecting mirror, a supporting structure, a tracking transmission mechanism, a control system and the like, is one of key components in a tower type solar thermal power generation system, and is also a main investment part of a power station. The solar photovoltaic power generation system has the functions of realizing the optimal tracking of solar radiation energy through the tracking control device, so that the solar radiation energy is accurately focused and reflected into a window of a heat absorber, the required solar energy is provided for the whole power generation system, and the solar photovoltaic power generation system is the basis for realizing solar thermal power generation.
Large tower-type photothermal power systems typically deploy thousands to tens of thousands of heliostats. During operation of the photothermal power generation system, each heliostat is independently controlled and tracks solar radiant energy. Due to various sources of tracking error, accurate alignment of all heliostats to 1mrad or less relative to a target point within the absorber window without a calibration system is considered impractical.
Disclosure of Invention
The invention provides a heliostat detection method of a tower type photothermal power station heliostat field based on a multi-rotor unmanned aerial vehicle, which has the defects of high equipment cost, high later maintenance cost, general detection efficiency and the like in the prior heliostat detection methods, such as a method for installing a camera on the ground, a method for installing a camera on a tower, a method for measuring central laser or radar, a method for detecting the position of a solar focal point by using a camera or a sensor on a tower and a method for installing a camera or a sensor on each heliostat, three-dimensional reconstruction of a mirror field is carried out by using a motion recovery structure algorithm through a picture sequence obtained by photographing of the multi-rotor unmanned aerial vehicle, information such as an included angle between the heliostat and a horizontal plane is obtained through calculation, the detection efficiency of the mirror field of the photo-thermal power station is improved while the higher detection precision is kept, and the heliostat in the mirror field of the photo-thermal power station can be ensured to obtain the higher detection precision.
The technical scheme adopted by the invention for solving the technical problems is as follows: a tower type photo-thermal power station mirror field inspection method based on a multi-rotor unmanned aerial vehicle comprises the following steps:
step 1, performing internal reference calibration on a multi-rotor unmanned aerial vehicle lens by using a Zhang chessboard method;
step 3, image preprocessing is carried out on an image sequence obtained by shooting of the multi-rotor unmanned aerial vehicle, and interference factors (such as trees, roads and the like in the image) in the image are removed;
step 4, extracting the feature points of the heliostat in each image for each preprocessed image, and performing feature matching in different image sequences with the same mirror to realize three-dimensional reconstruction to obtain a three-dimensional model of the heliostat in a mirror field;
and 5, calculating an included angle between the heliostat and an installation horizontal plane in the heliostat field through the obtained three-dimensional model of the heliostat, and realizing the patrol of the multi-rotor unmanned aerial vehicle on the position and the pose of the heliostat in the heliostat field.
Further, in step 1, the step of performing internal reference calibration on the multi-rotor unmanned aerial vehicle lens by using the zhang's chessboard method is as follows:
step 1-1, shooting the A4 paper printed with the chessboard pictures by using a multi-rotor unmanned aerial vehicle lens to obtain photographic images of the chessboard pictures at different angles;
step 1-2, importing the photographic image into MATLAB, and utilizing a chessboard method calibration tool box of the MATLAB to carry out solving, and calibrating the internal parameters of the multi-rotor unmanned aerial vehicle lens.
Further, in step 2, a multi-rotor unmanned aerial vehicle is used for cruising and shooting above a mirror field of the tower type photo-thermal power station, and the step of obtaining an image sequence of the mirror field is as follows:
step 2-1, designing a cruising route of the multi-rotor unmanned aerial vehicle above a mirror field, wherein the cruising route is designed to be S-shaped, so that a complete mirror field image can be shot on the cruising route, and simultaneously designing a time interval (such as shooting an image every 0.5 seconds) for shooting by the multi-rotor unmanned aerial vehicle and a shooting angle (namely an inclined angle formed by a lens and a horizontal plane, such as 30 degrees) for shooting by the multi-rotor unmanned aerial vehicle;
and 2-2, operating the multi-rotor unmanned aerial vehicle to cruise and fly above the mirror field according to a preset cruise route, wherein the single flight time of the multi-rotor unmanned aerial vehicle is 30-35 minutes, and shooting is carried out according to a preset time interval and an angle in the cruise process, so that a picture sequence of the mirror field is obtained.
Further, in step 3, the step of performing image preprocessing on the image sequence obtained by shooting by the multi-rotor unmanned aerial vehicle is as follows:
step 3-1, converting the original image into a gray-scale image, wherein the step uses a gray-scale image conversion function provided by OpenCV;
3-2, selecting a proper threshold value through an Otsu threshold value method, separating the foreground and the distant view of the image, and generating a mask;
3-3, processing the mask by adopting an image morphology method, namely performing expansion and corrosion operations, wherein partial interference still exists in the mask; performing image morphology opening operation, namely corroding and then expanding the image to eliminate small objects; performing image morphological closed operation, namely performing expansion and then corrosion on the image, and removing small black holes; performing image morphology open operation on the initial mask by using a circular nuclear operator with the size of 11 x 11 to eliminate small object interference in the image, and performing closed operation on the open operation result by using a circular nuclear operator with the size of 3 x 3 to fill up the internal cavity of the object in the image;
and 3-4, performing bit operation on the processed mask and the original image to obtain a mirror field image with clear background (interference items such as grassland, trees, roads and the like are removed).
Further, in step 4, extracting feature points of the heliostat in each image, and performing feature matching in a collection of different image sequences in which the same mirror appears to realize three-dimensional reconstruction, so as to obtain a three-dimensional model of the heliostat in the mirror field, the steps are as follows:
step 4-1, inputting image frame I at current time point Ii，i≥1，
wherein ,xjRepresentative image IiCharacteristic point of (1), fjA description of the appearance of the representative feature points,
representative image IiTotal number of feature points in (1), FiA set of representative feature points and their descriptions;
step 4-2, for image frame Ia and In，C＝{(Ia，Ib)|Ia，IbThe method belongs to I, a is less than b, wherein I represents the current image sequence, and the characteristic matching is carried out by adopting an SIFT characteristic matching algorithm, and the method comprises the following steps: for the input image sequence I, according to the characteristic points in the step 4-1 and the described set F thereofiPerforming feature matching between the two images to obtain matching points, and finally outputting an image pair set;
step 4-3, according to the matching points obtained in the step 4-2, using a function findEsentialMat () provided in OpenCV to obtain an eigen matrix; decomposing the intrinsic matrix by using OpenCV, and calculating to obtain a rotation matrix R and a displacement matrix T between two corresponding cameras of the two images;
step 4-4, now knowing the rotation matrix R and the displacement matrix T of the camera between the two image frames, and also the coordinates of each pair of matching points; the three-dimensional reconstruction is to restore the coordinates of the matching points in the space through the known information and solve the coordinates by using a space triangulation method, so that the space coordinates of the heliostat feature points can be obtained;
step 4-5, the spatial triangulation method in step 4-4 solves the space coordinate of the heliostat feature point and has larger error, the invention introduces a light beam adjustment method to optimize the solving process, and optimizes the loss function of the light beam adjustment method, specifically: introducing heliostat geometry and GPS camera coordinates into a loss function, and for an M-frame image sequence, wherein each frame comprises N characteristic points, expressing the loss function of a beam adjustment method as follows:
wherein ,
representing the estimated coordinates of the points,. pi.represents the projection function in space, if the point j is on the ith frame image, then thetaij1, otherwise θij＝0，Ri，Ti，XjRespectively representing the spatial coordinates of an ith frame image rotation matrix, an ith frame image displacement matrix and an ith frame image point j; k represents the number of side lengths of the heliostat, eta represents a compensation coefficient, a'iRepresenting the length of a side of a heliostat obtained by three-dimensional reconstruction, aiRepresenting the true side length of the heliostat, Huber (a'i-ai) A loss function representing the difference therebetween; λ represents a compensation coefficient, CiRepresenting the coordinates of the three-dimensional points of the camera obtained by three-dimensional reconstruction,
a coordinate point representing a record of GPS information of the camera,
a loss function representing the difference therebetween; the Huber loss function has the characteristics of smoothing average absolute errors, having better robustness on noise and outliers and avoiding mutation caused by too large or too small estimation values; and a beam adjustment method is introduced, and a loss function of the beam adjustment method is optimized, so that the error of the space coordinate of the heliostat feature point is reduced, and the precision of three-dimensional reconstruction is improved.
Further, in step 5, calculating an included angle between the heliostat and a horizontal plane in the heliostat field through the obtained three-dimensional model of the heliostat as follows:
step 5-1, obtaining the spatial coordinates of the heliostat feature points through step 4, and calculating the spatial coordinates of the geometric center of the heliostat;
step 5-2, calculating the installation horizontal plane of the heliostat according to the geometric center coordinates of the multi-surface heliostat;
and 5-3, calculating the included angle between the heliostat and the installation horizontal plane through the plane of the characteristic point of the heliostat and the installation horizontal plane of the geometric center of the heliostat.
The invention also provides a tower type photothermal power station scope inspection device based on the multi-rotor unmanned aerial vehicle, which comprises a memory and one or more processors, wherein executable codes are stored in the memory, and when the processors execute the executable codes, the tower type photothermal power station scope inspection device based on the multi-rotor unmanned aerial vehicle is used for realizing the tower type photothermal power station scope inspection method based on the multi-rotor unmanned aerial vehicle.
The invention has the following beneficial effects: this tower light and heat power station mirror field inspection method based on many rotor unmanned aerial vehicle provides a neotype detection method of light and heat power station mirror field for the detection cost of light and heat power station heliostat reduces, and detection efficiency promotes, and the detection precision such as the angle of heliostat is higher.
Drawings
FIG. 1 is a flow chart of a tower type photo-thermal power station mirror field inspection method based on a multi-rotor unmanned aerial vehicle;
FIG. 2 is a flowchart of three-dimensional reconstruction of a mirror field and calculation of an included angle between a heliostat and an installation horizontal plane in the mirror field according to an embodiment of the present invention;
FIG. 3 is a comparison graph of the optimization results of the loss function by the beam adjustment method according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of an included angle between a heliostat mirror surface and an installation horizontal plane in the embodiment of the invention;
FIG. 5 is a structural block diagram of the tower type photothermal power station mirror field inspection device based on the multi-rotor unmanned aerial vehicle.
Detailed Description
For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.
It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
Referring to fig. 1 and 2, the method for inspecting the mirror field of the tower type photothermal power station based on the multi-rotor unmanned aerial vehicle provided by the embodiment includes the following steps:
step 1, performing internal reference calibration on a multi-rotor unmanned aerial vehicle lens by using a Zhang chessboard method; the method comprises the following specific steps:
step 1-1, shooting the A4 paper printed with the chessboard pictures by using a multi-rotor unmanned aerial vehicle lens to obtain photographic images of the chessboard pictures at different angles;
step 1-2, importing the photographic image into MATLAB, and utilizing a chessboard method calibration tool box existing in the MATLAB to carry out solving, and calibrating the internal reference of the multi-rotor unmanned aerial vehicle lens; the method comprises the following specific steps:
step 2-1, designing a cruising route of the multi-rotor unmanned aerial vehicle above a mirror field, wherein the cruising route is designed to be S-shaped, so that a complete mirror field image can be shot on the cruising route, and simultaneously designing a time interval (such as shooting an image every 0.5 seconds) for shooting by the multi-rotor unmanned aerial vehicle and a shooting angle (namely an inclined angle formed by a lens and a horizontal plane, such as 30 degrees) for shooting by the multi-rotor unmanned aerial vehicle;
and 2-2, operating the multi-rotor unmanned aerial vehicle to cruise and fly above the mirror field according to a preset cruise route, wherein the single flight time of the multi-rotor unmanned aerial vehicle is 30-35 minutes, and shooting is carried out according to a preset time interval and an angle in the cruise process, so that a picture sequence of the mirror field is obtained.
step 3, image preprocessing is carried out on an image sequence obtained by shooting of the multi-rotor unmanned aerial vehicle, and interference factors (such as trees, roads and the like in the image) in the image are removed; the method comprises the following specific steps:
step 3-1, converting the original image into a gray-scale image, wherein the step uses a gray-scale image conversion function provided by OpenCV;
3-2, selecting a proper threshold value through an Otsu threshold value method, separating the foreground and the distant view of the image, and generating a mask;
3-3, processing the mask by adopting an image morphology method, namely performing expansion and corrosion operations, wherein partial interference still exists in the mask; performing image morphology opening operation, namely corroding and then expanding the image to eliminate small objects; performing image morphological closed operation, namely performing expansion and then corrosion on the image, and removing small black holes; performing image morphology open operation on the initial mask by using a circular nuclear operator with the size of 11 x 11 to eliminate small object interference in the image, and performing closed operation on the open operation result by using a circular nuclear operator with the size of 3 x 3 to fill up the internal cavity of the object in the image;
and 3-4, performing bit operation on the processed mask and the original image to obtain a mirror field image with clear background (interference items such as grassland, trees, roads and the like are removed).
Step 4, extracting the feature points of the heliostat in each image for each preprocessed image, and performing feature matching in different image sequences with the same mirror to realize three-dimensional reconstruction to obtain a three-dimensional model of the heliostat in a mirror field; the method comprises the following specific steps:
step 4-1, inputting image frame I at current time point Ii，i≥1，
wherein ,xjRepresentative image IiCharacteristic point of (1), fjA description of the appearance of the representative feature points,
representative image IiTotal number of feature points in (1), FiRepresentative feature point and itsA set of descriptions;
step 4-2, for image frame Ia and Ib，C＝{(Ia，Ib)|Ia，IbThe method belongs to I, a is less than b, wherein I represents the current image sequence, and the characteristic matching is carried out by adopting an SIFT characteristic matching algorithm, and the method comprises the following steps: for the input image sequence I, according to the characteristic points in the step 4-1 and the described set F thereofiPerforming feature matching between the two images to obtain matching points, and finally outputting an image pair set;
step 4-3, according to the matching points obtained in the step 4-2, using a function findEsentialMat () provided in OpenCV to obtain an eigen matrix; decomposing the intrinsic matrix by using OpenCV, and calculating to obtain a rotation matrix R and a displacement matrix T between two corresponding cameras of the two images;
step 4-4, now knowing the rotation matrix R and the displacement matrix T of the camera between the two image frames, and also the coordinates of each pair of matching points; the three-dimensional reconstruction is to restore the coordinates of the matching points in the space through the known information and solve the coordinates by using a space triangulation method, so that the space coordinates of the heliostat feature points can be obtained;
step 4-5, the spatial triangulation method in step 4-4 solves the space coordinate of the heliostat feature point and has larger error, the invention introduces a light beam adjustment method to optimize the solving process, and optimizes the loss function of the light beam adjustment method, specifically: introducing heliostat geometry and GPS camera coordinates into a loss function, and for an M-frame image sequence, wherein each frame comprises N characteristic points, expressing the loss function of a beam adjustment method as follows:
wherein ,
representing the estimated coordinates of the points,. pi.represents the projection function in space, if the point j is on the ith frame image, then thetaij1, otherwise θij＝0，Ri，Yi，XjRespectively representing the spatial coordinates of an ith frame image rotation matrix, an ith frame image displacement matrix and an ith frame image point j; k represents the number of side lengths of the heliostat, eta represents a compensation coefficient, a'iRepresenting the length of a side of a heliostat obtained by three-dimensional reconstruction, aiRepresenting the true side length of the heliostat, Huber (a'i-ai) A loss function representing the difference therebetween; λ represents a compensation coefficient, CiRepresenting the coordinates of the three-dimensional points of the camera obtained by three-dimensional reconstruction,
a coordinate point representing a record of GPS information of the camera,
a loss function representing the difference therebetween; the Huber loss function has the characteristics of smoothing average absolute errors, having better robustness on noise and outliers and avoiding mutation caused by too large or too small estimation values; and a beam adjustment method is introduced, and a loss function of the beam adjustment method is optimized, so that the error of the space coordinate of the heliostat feature point is reduced, and the precision of three-dimensional reconstruction is improved.
As shown in fig. 3, the error of the side length of the heliostat after three-dimensional reconstruction is reduced from 0.0779m to 0.0491m by the beam adjustment method, and the arithmetic mean of the error of the side length of the heliostat after three-dimensional reconstruction is reduced from 0.0491m to 0.0390m by introducing the heliostat geometry and the GPS camera coordinates into the loss function of the beam adjustment method.
Step 5, calculating an included angle between the heliostat and an installation horizontal plane in the heliostat field through the obtained three-dimensional model of the heliostat, and realizing the patrol of the multi-rotor unmanned aerial vehicle on the position and posture of the heliostat in the heliostat field; the method comprises the following specific steps:
step 5-1, obtaining the spatial coordinates of the heliostat feature points through step 4, and calculating the spatial coordinates of the geometric center of the heliostat;
step 5-2, calculating the installation horizontal plane of the heliostat according to the geometric center coordinates of the multi-surface heliostat;
and 5-3, calculating the included angle between the heliostat and the installation horizontal plane through the plane of the heliostat feature point and the installation horizontal plane of the geometric center of the heliostat, as shown in fig. 4, showing a schematic diagram of the included angle between the plane of the 28-surface heliostat feature point in the heliostat field and the installation horizontal plane of the geometric center of the heliostat, inspecting the whole heliostat field (100-surface heliostats in total) through a multi-rotor type unmanned aerial vehicle, wherein the number of the correctly detected heliostats is 67 surfaces, the correct detection rate is 67%, the average calculation value of the heliostat included angles in the heliostat field is 29.975mrad, the error of the included angle with the preset 30 degrees is 0.436mrad, and the requirement of 0.5mrad is met.
Corresponding to the embodiment of the tower type photothermal power station scope inspection method based on the multi-rotor unmanned aerial vehicle, the invention also provides an embodiment of a tower type photothermal power station scope inspection device based on the multi-rotor unmanned aerial vehicle.
Referring to fig. 5, the tower type photothermal power station scope inspection device based on the multi-rotor unmanned aerial vehicle provided by the embodiment of the invention comprises a memory and one or more processors, wherein executable codes are stored in the memory, and when the processors execute the executable codes, the tower type photothermal power station scope inspection device based on the multi-rotor unmanned aerial vehicle is used for realizing the tower type photothermal power station scope inspection method based on the multi-rotor unmanned aerial vehicle in the embodiment.
The embodiment of the tower type photothermal power station mirror field inspection device based on the multi-rotor unmanned aerial vehicle can be applied to any equipment with data processing capacity, and the any equipment with data processing capacity can be equipment or devices such as computers. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory for running through the processor of any device with data processing capability. In terms of hardware, as shown in fig. 5, a hardware structure diagram of any device with data processing capability where the tower photothermal power station mirror field inspection device based on the multi-rotor unmanned aerial vehicle of the present invention is located is shown, except for the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 5, in the embodiment, any device with data processing capability where the device is located may also include other hardware according to the actual function of the any device with data processing capability, which is not described again.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
The embodiment of the invention also provides a computer-readable storage medium, wherein a program is stored on the computer-readable storage medium, and when the program is executed by a processor, the method for inspecting the mirror field of the tower type photo-thermal power station based on the multi-rotor unmanned aerial vehicle is realized.
The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any data processing capability device described in any of the foregoing embodiments. The computer readable storage medium may also be any external storage device of a device with data processing capabilities, such as a plug-in hard disk, a Smart Media Card (SMC), an SD Card, a Flash memory Card (Flash Card), etc. provided on the device. Further, the computer readable storage medium may include both an internal storage unit and an external storage device of any data processing capable device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the arbitrary data processing-capable device, and may also be used for temporarily storing data that has been output or is to be output.
The above embodiments are merely illustrative of the present invention and are not to be construed as limiting the invention. Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that various combinations, modifications or equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention, and the technical solution of the present invention is covered by the claims of the present invention.
Claims (8)
1. The tower type photothermal power station mirror field inspection method based on the multi-rotor unmanned aerial vehicle is characterized by comprising the following steps of:
step 1, performing internal reference calibration on a multi-rotor unmanned aerial vehicle lens by using a Zhang chessboard method;
step 2, performing cruise shooting on the tower type photo-thermal power station mirror field by using a multi-rotor unmanned aerial vehicle to obtain an image sequence of the mirror field;
step 3, image preprocessing is carried out on an image sequence obtained by shooting of the multi-rotor unmanned aerial vehicle, and interference factors in the image are removed;
step 4, extracting the feature points of the heliostat in each image for each preprocessed image, and performing feature matching in different image sequences with the same mirror to realize three-dimensional reconstruction to obtain a three-dimensional model of the heliostat in a mirror field;
and 5, calculating an included angle between the heliostat and an installation horizontal plane in the heliostat field through the obtained three-dimensional model of the heliostat, and realizing the patrol of the multi-rotor unmanned aerial vehicle on the position and the pose of the heliostat in the heliostat field.
2. The method according to claim 1, wherein in step 1, the step of performing internal reference calibration on the lens of the multi-rotor unmanned aerial vehicle by using the Zhang chessboard method comprises the following steps:
step 1-1, shooting the A4 paper printed with the chessboard pictures by using a multi-rotor unmanned aerial vehicle lens to obtain photographic images of the chessboard pictures at different angles;
step 1-2, importing the photographic image into MATLAB, and utilizing a chessboard method calibration tool box of the MATLAB to carry out solving, and calibrating the internal parameters of the multi-rotor unmanned aerial vehicle lens.
3. The method according to claim 1, wherein in the step 2, a multi-rotor unmanned aerial vehicle is used for cruise shooting above a mirror field of the tower type photothermal power station, and the step of obtaining the image sequence of the mirror field is as follows:
step 2-1, designing a cruising route of the multi-rotor unmanned aerial vehicle above a mirror field, wherein the cruising route is designed to be S-shaped, so that a complete mirror field image can be shot on the cruising route, and simultaneously designing the shooting time interval of the multi-rotor unmanned aerial vehicle and the shooting angle of the multi-rotor unmanned aerial vehicle;
and 2-2, operating the multi-rotor unmanned aerial vehicle to cruise and fly above the mirror field according to a preset cruise route, and shooting according to a preset time interval and an angle in the cruise process, so that a picture sequence of the mirror field is obtained.
4. The method according to claim 1, wherein in step 3, the step of image preprocessing the sequence of images captured by the multi-rotor drone is as follows:
step 3-1, converting the original image into a gray-scale image;
3-2, selecting a proper threshold value through an Otsu threshold value method, separating the foreground and the distant view of the image, and generating a mask;
3-3, processing the mask by adopting an image morphology method, performing image morphology opening operation on the initial mask by using a circular nuclear operator with the size of 11 x 11 to eliminate small object interference in the image, and performing closing operation on the opening operation result by using the circular nuclear operator with the size of 3 x 3 to fill the inner cavity of the object in the image;
and 3-4, performing bit operation on the processed mask and the original image to obtain a mirror field image with clear background.
5. The method according to claim 1, wherein in step 4, the step of extracting the feature points of the heliostat in each image, and performing feature matching in different image sequence collections in which the same mirror appears to realize three-dimensional reconstruction to obtain a three-dimensional model of the heliostat in the mirror field comprises the following steps:
step 4-1, inputting image frame I at current time point Ii，i≥1，
wherein ,xjRepresentative image IiCharacteristic point of (1), fjA description of the appearance of the representative feature points,
representative image IiTotal number of feature points in (1), FiA set of representative feature points and their descriptions;
step 4-2, for image frame Ia and Ib，C＝{(Ia,Ib)|Ia,IbThe method belongs to I, a is less than b, wherein I represents the current image sequence, and the characteristic matching is carried out by adopting an SIFT characteristic matching algorithm, and the method comprises the following steps: for the input image sequence I, according to the characteristic points in the step 4-1 and the described set F thereofiPerforming feature matching between the two images to obtain matching points, and finally outputting an image pair set;
step 4-3, according to the matching points obtained in the step 4-2, using a function findEsentialMat () provided in OpenCV to obtain an eigen matrix; decomposing the intrinsic matrix by using OpenCV, and calculating to obtain a rotation matrix R and a displacement matrix T between two corresponding cameras of the two images;
and 4-4, performing three-dimensional reconstruction according to the rotation matrix R and the displacement matrix T of the camera between the two image frames and the coordinates of each pair of matching points, restoring the coordinates of the matching points in the space, and solving by using a space triangulation method to obtain the space coordinates of the heliostat feature points.
6. The method according to claim 5, wherein in step 4-4, a beam balancing method is introduced to optimize the solving process and optimize the loss function of the beam balancing method, specifically: introducing heliostat geometry and GPS camera coordinates into a loss function, and for an M-frame image sequence, wherein each frame comprises N characteristic points, expressing the loss function of a beam adjustment method as follows:
wherein ,
representing the estimated coordinates of the points,. pi.represents the projection function in space, if the point j is on the ith frame image, then thetaij1, otherwise θij＝0，Ri,Ti,XjRespectively representing the spatial coordinates of an ith frame image rotation matrix, an ith frame image displacement matrix and an ith frame image point j; k represents the number of side lengths of the heliostat, eta represents a compensation coefficient, a'iRepresenting the length of a side of a heliostat obtained by three-dimensional reconstruction, aiRepresenting the true side length of the heliostat, Huber (a'i-ai) A loss function representing the difference therebetween; λ represents a compensation coefficient, CiRepresenting the coordinates of the three-dimensional points of the camera obtained by three-dimensional reconstruction,
a coordinate point representing a record of GPS information of the camera,
a loss function representing the difference therebetween.
7. The method according to claim 1, wherein in the step 5, the step of calculating the angle between the heliostat and the horizontal plane in the heliostat field through the obtained three-dimensional model of the heliostat is as follows:
step 5-1, obtaining the spatial coordinates of the heliostat feature points through step 4, and calculating the spatial coordinates of the geometric center of the heliostat;
step 5-2, calculating the installation horizontal plane of the heliostat according to the geometric center coordinates of the multi-surface heliostat;
and 5-3, calculating the included angle between the heliostat and the installation horizontal plane through the plane of the characteristic point of the heliostat and the installation horizontal plane of the geometric center of the heliostat.
8. A tower type photothermal power station scope inspection device based on a multi-rotor unmanned aerial vehicle comprises a memory and one or more processors, and is characterized in that executable codes are stored in the memory, and when the executable codes are executed by the processors, the tower type photothermal power station scope inspection device based on the multi-rotor unmanned aerial vehicle is used for realizing the tower type photothermal power station scope inspection method based on the multi-rotor unmanned aerial vehicle according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210003230.8A CN114355977B (en) | 2022-01-04 | 2022-01-04 | Tower type photo-thermal power station mirror field inspection method and device based on multi-rotor unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210003230.8A CN114355977B (en) | 2022-01-04 | 2022-01-04 | Tower type photo-thermal power station mirror field inspection method and device based on multi-rotor unmanned aerial vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114355977A true CN114355977A (en) | 2022-04-15 |
| CN114355977B CN114355977B (en) | 2023-09-22 |
Family
ID=81106098
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210003230.8A Active CN114355977B (en) | 2022-01-04 | 2022-01-04 | Tower type photo-thermal power station mirror field inspection method and device based on multi-rotor unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114355977B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831652A (en) * | 2012-08-21 | 2012-12-19 | 南京大学 | Automatic ticket checking device capable of carrying out passing detection on the basis of machine vision |
| CN109238173A (en) * | 2018-08-16 | 2019-01-18 | 中国矿业大学 | A kind of coal yard three-dimensional live reconfiguration system and coal amount Method of fast estimating |
| CN110398233A (en) * | 2019-09-04 | 2019-11-01 | 浙江中光新能源科技有限公司 | A kind of heliostat field coordinate mapping system and method based on unmanned plane |
| CN110716576A (en) * | 2019-11-07 | 2020-01-21 | 浙江中光新能源科技有限公司 | Heliostat field inspection system and method based on unmanned aerial vehicle |
| US20200134847A1 (en) * | 2018-10-31 | 2020-04-30 | Bentley Systems, Incorporated | Structure depth-aware weighting in bundle adjustment |
| CN112102458A (en) * | 2020-08-31 | 2020-12-18 | 湖南盛鼎科技发展有限责任公司 | Single-lens three-dimensional image reconstruction method based on laser radar point cloud data assistance |
| CN112288875A (en) * | 2020-10-30 | 2021-01-29 | 中国有色金属长沙勘察设计研究院有限公司 | Rapid three-dimensional reconstruction method for unmanned aerial vehicle mine inspection scene |
| CN112465984A (en) * | 2020-11-12 | 2021-03-09 | 西北工业大学 | Monocular camera sequence image three-dimensional reconstruction method based on double-layer filtering |
| CN112560722A (en) * | 2020-12-22 | 2021-03-26 | 中国人民解放军国防科技大学 | Airplane target identification method and device, computer equipment and storage medium |
| CN112561875A (en) * | 2020-12-14 | 2021-03-26 | 孙猛猛 | Photovoltaic cell panel coarse grid detection method based on artificial intelligence |
-
2022
- 2022-01-04 CN CN202210003230.8A patent/CN114355977B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831652A (en) * | 2012-08-21 | 2012-12-19 | 南京大学 | Automatic ticket checking device capable of carrying out passing detection on the basis of machine vision |
| CN109238173A (en) * | 2018-08-16 | 2019-01-18 | 中国矿业大学 | A kind of coal yard three-dimensional live reconfiguration system and coal amount Method of fast estimating |
| US20200134847A1 (en) * | 2018-10-31 | 2020-04-30 | Bentley Systems, Incorporated | Structure depth-aware weighting in bundle adjustment |
| CN110398233A (en) * | 2019-09-04 | 2019-11-01 | 浙江中光新能源科技有限公司 | A kind of heliostat field coordinate mapping system and method based on unmanned plane |
| CN110716576A (en) * | 2019-11-07 | 2020-01-21 | 浙江中光新能源科技有限公司 | Heliostat field inspection system and method based on unmanned aerial vehicle |
| CN112102458A (en) * | 2020-08-31 | 2020-12-18 | 湖南盛鼎科技发展有限责任公司 | Single-lens three-dimensional image reconstruction method based on laser radar point cloud data assistance |
| CN112288875A (en) * | 2020-10-30 | 2021-01-29 | 中国有色金属长沙勘察设计研究院有限公司 | Rapid three-dimensional reconstruction method for unmanned aerial vehicle mine inspection scene |
| CN112465984A (en) * | 2020-11-12 | 2021-03-09 | 西北工业大学 | Monocular camera sequence image three-dimensional reconstruction method based on double-layer filtering |
| CN112561875A (en) * | 2020-12-14 | 2021-03-26 | 孙猛猛 | Photovoltaic cell panel coarse grid detection method based on artificial intelligence |
| CN112560722A (en) * | 2020-12-22 | 2021-03-26 | 中国人民解放军国防科技大学 | Airplane target identification method and device, computer equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| CARLOS CAMPOS: "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM", 《IEEE TRANSACTIONS ON ROBOTICS 》 * |
| CESAR CADENA: "Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age", 《 IEEE TRANSACTIONS ON ROBOTICS 》 * |
| 匡文彬: "基于固定式相机对动态目标跟踪的三维重建", 《计算技术与自动化》 * |
| 吴海滨: "基于计算机视觉的人体内腔三维重建技术综述", 《J计算机工程》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114355977B (en) | 2023-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111968129B (en) | Instant positioning and map construction system and method with semantic perception | |
| CN113538391B (en) | Photovoltaic defect detection method based on Yolov4 and thermal infrared image | |
| CN107564062B (en) | Pose abnormity detection method and device | |
| CN110514298B (en) | Solar radiation intensity calculation method based on foundation cloud picture | |
| CN110619678B (en) | Electric power tower inspection track generation method based on point cloud matching | |
| CN113159466B (en) | Short-time photovoltaic power generation prediction system and method | |
| CN114529817A (en) | Unmanned aerial vehicle photovoltaic fault diagnosis and positioning method based on attention neural network | |
| Huang et al. | Correlation and local feature based cloud motion estimation | |
| CN113298947A (en) | Multi-source data fusion-based three-dimensional modeling method medium and system for transformer substation | |
| Sampath et al. | Estimation of rooftop solar energy generation using satellite image segmentation | |
| CN111899345B (en) | Three-dimensional reconstruction method based on 2D visual image | |
| CN116449384A (en) | Radar inertial tight coupling positioning mapping method based on solid-state laser radar | |
| CN114283137A (en) | Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network | |
| CN114140527A (en) | Dynamic environment binocular vision SLAM method based on semantic segmentation | |
| CN117058396A (en) | Fan blade defect area rapid segmentation method and system based on artificial intelligence | |
| WO2024037123A1 (en) | Full-field refined dni prediction method | |
| Huang et al. | Life: Lighting invariant flow estimation | |
| CN113326752A (en) | Unmanned aerial vehicle-based photovoltaic power station identification method and system | |
| Li et al. | Monocular 3-D Object Detection Based on Depth-Guided Local Convolution for Smart Payment in D2D Systems | |
| CN114355977B (en) | Tower type photo-thermal power station mirror field inspection method and device based on multi-rotor unmanned aerial vehicle | |
| Özer et al. | An approach based on deep learning methods to detect the condition of solar panels in solar power plants | |
| CN116524382A (en) | Bridge swivel closure accuracy inspection method system and equipment | |
| CN107194334B (en) | Video satellite image dense Stereo Matching method and system based on optical flow estimation | |
| WO2022252362A1 (en) | Geometry and texture based online matching optimization method and three-dimensional scanning system | |
| CN114092838A (en) | Photovoltaic defect detection method based on deep learning target detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |