CN115586789A

CN115586789A - Vision-based method for confirming and inspecting suspension point of wind driven generator inspection unmanned aerial vehicle

Info

Publication number: CN115586789A
Application number: CN202211507946.8A
Authority: CN
Inventors: 肖玄; 杨润; 李成
Original assignee: Chengdu Rainpoo Technology Co ltd
Current assignee: Chengdu Rainpoo Technology Co ltd
Priority date: 2022-11-29
Filing date: 2022-11-29
Publication date: 2023-01-10
Anticipated expiration: 2042-11-29
Also published as: CN115586789B

Abstract

The invention discloses a vision-based unmanned aerial vehicle suspension point confirming and inspecting method for inspecting a wind driven generator. The inspection method is based on the suspension point determined by the suspension point determination method. According to the scheme, through a pure visual load scheme of the unmanned aerial vehicle, under the condition that the wind driven generator does not stop, the dead ahead and/or dead behind of the wind driven generator are determined, and the current yaw angle of the wind driven generator is analyzed; therefore, when the confirmed suspension point is used for inspection, blade image identification and image feature definition are facilitated.

Description

Vision-based method for confirming and inspecting suspension point of wind driven generator inspection unmanned aerial vehicle

Technical Field

The invention relates to the technical field of aerial photography, in particular to a vision-based method for confirming and inspecting a suspension point of an unmanned aerial vehicle by inspecting a wind driven generator.

Background

The wind driven generator is equipment for converting wind energy into electric energy, is an important component in the field of new energy power generation, and has become the fastest growing wind power development market in China worldwide. The wind power generator generally comprises blades, a generator, a direction regulator, a tower, a speed-limiting safety framework, an energy storage device and the like. The blades are the most basic and the most critical and expensive components in the wind driven generator, and the good performance of the blades is the first factor for ensuring the normal and stable operation of the unit.

Because the whole work that exposes in natural environment of blade receives the sand blown by the wind to corrode, uses defects such as surface wear, perforation generally all can appear after certain years, and this not only influences the generating efficiency, appears the potential safety hazard even when serious: when the blades are in failure accidents, particularly single-blade fracture accidents, the balance state among the blades is damaged, the vibration of the generator set is suddenly intensified, if no unit protection measures exist or the unit protection is not timely intervened in the state, serious damage is brought to a generator set shaft system and a tower drum, and the whole generator set is possibly damaged in serious conditions. In the long-term operation process of the wind driven generator, the blade usually needs a long-term process when being out of work, for example, the damage condition of the surface of the blade can be effectively monitored in the early stage, and serious accidents such as the blade fracture phenomenon can be effectively avoided. Therefore, regular inspection and maintenance of the wind driven generator are important components of operation and maintenance of the wind driven generator.

At present, carry out the comparatively common mode of damage detection to the blade surface and include: the technical scheme includes technical schemes provided by patent application documents with application numbers of CN202011275910.2, CN201911393629.6, CN202210088479.3, CN202010115183.7, CN201910753262.8, CN202110643559.6, CN202011275910.2 and the like.

Nowadays, it is fairly common to use unmanned aerial vehicle to carry out aerogenerator and patrol and examine, and the future trend is to let unmanned aerial vehicle realize independently, intelligent automation patrols and examines completely. Under the general condition, according to the mode of patrolling and examining, carry out aerogenerator with unmanned aerial vehicle and patrol and examine and need confirm and shoot position/aerogenerator characteristic position. Regarding the above position determination, the prior art discloses: CN110910349A, the name of inventive creation is: a wind turbine state obtaining method based on aerial photography vision; the publication numbers are: CN113759960A, inventive name: unmanned aerial vehicle-based fan blade and tower drum patrol inspection identification systems and methods both disclose relevant methods, particularly for confirming a hub central point of a wind driven generator.

The detection technology for detecting the surface damage of the wind driven generator blade is further optimized, and the method has positive significance for promoting the development process of clean energy in China undoubtedly.

Disclosure of Invention

Aiming at the technical problems that the detection technology for detecting the surface damage of the wind driven generator blade is further optimized and has positive significance for promoting the development process of clean energy in China undoubtedly, the invention provides a visual-based method for confirming and inspecting the suspension point of the wind driven generator inspection unmanned aerial vehicle. The technical scheme that this scheme provided can be used to aerogenerator patrols and examines unmanned aerial vehicle and patrol and examine the point location of hovering and confirm, specifically for determining behind aerogenerator's dead ahead and/or dead back, determine again and patrol and examine unmanned aerial vehicle image and acquire the suspension point.

Aiming at the problems, the method for confirming the suspension point of the unmanned aerial vehicle for routing inspection of the wind driven generator based on the vision solves the problems through the following technical points: a vision-based method for confirming a suspension point of an unmanned aerial vehicle during inspection of a wind driven generator is characterized in that the suspension point of the unmanned aerial vehicle during inspection is confirmed to be positioned right above the central line of a hub of the wind driven generator by confirming the right front and/or the right back of the wind driven generator;

wherein, the positive front and/or positive back confirmation method is as follows:

s1, confirming an annular air route of unmanned aerial vehicle flight according to the position coordinate of the wind driven generator and the height of the central line of the hub: the annular route is positioned on the outer side of the rotating track of the wind driven generator blade, the annular route is positioned on a horizontal plane at the height, and the position coordinate is positioned on the central line of the annular route;

s2, shooting the blades of the wind driven generator by the unmanned aerial vehicle in the flying process along the circular route, wherein the circular route is provided with a plurality of shooting points, and when the view field of the camera is towards one side of the wind driven generator, in the formed picture, even if the camera on the circular route is positioned right in front of or right behind the wind driven generator, different blades can be caused to present different visual lengths in the picture, so that the length acquired in the step S3 is not used for calculating the optimal length, and finally the accuracy of right in front of or right behind is influenced, and on the basis, the optimal length is preferably set as follows: the lens of each shooting point faces to the center of the annular route, and the whole blade is positioned in the photo imaging area of each shooting point;

s3, obtaining the length of each blade in each photo in the visual state, and determining the front or the back of the wind driven generator according to the difference value between the maximum blade length and the minimum blade length aiming at each photo;

and the shooting point corresponding to the picture with the minimum difference is positioned right in front of or right behind the wind driven generator.

In the prior art, according to a set inspection method of a wind driven generator, the position of a hub central point of the wind driven generator or a position of a aerial camera relative to the wind driven generator is determined as one of key technologies. For example, after obtaining the hub center point of the wind driven generator, the hovering shooting position of the aerial camera can be confirmed by using the hub center point of the wind driven generator as a feature point, and the obtaining manner of the hub center point of the wind driven generator includes: the hub center is obtained by utilizing the characteristic that the hub of the wind driven generator is round in the visual state of the main visual direction through image recognition, but when the view port is positioned at the rear side of the wind driven generator, the shape of the rear end of the engine room is not necessarily a regular figure, so that the central point of the rear end of the engine room cannot be directly obtained through the image recognition of the end part of the engine room in the prior art relatively accurately. In addition, the conventional shooting position confirmation also comprises the following steps of obtaining the shooting position of the aerial photography instrument: because the shooting position is related to the position of the wind driven generator and the yaw angle of the wind driven generator, firstly confirming the position and the yaw angle of the wind driven generator or parameters such as the right front/right back of the wind driven generator and the like, and then confirming the shooting position of the aerial photography instrument, wherein in the process, the position of the wind driven generator (such as the longitude and latitude of the installation position) is a known value, but the yaw angle is possibly changed in the working process of the wind driven generator, specific data generally exist in computer data of the wind driven generator and cannot be intuitively obtained, and based on the situation, an unmanned aerial vehicle flyer confirms the yaw angle in a visual mode, so that intelligent routing inspection implementation is not facilitated, and meanwhile, related results are greatly influenced by operators; other, current intelligence is patrolled and examined the scheme and is then mostly operated unmanned aerial vehicle earlier and fly to the aerogenerator dead ahead that has shut down, and rethread visual identification and laser radar build the picture and carry out intelligence and patrol and examine the step, and this makes aerogenerator patrol and examine the dependence stronger to other equipment.

Aiming at the aim of routing inspection of the wind driven generator, the scheme provides a method for confirming the suspension point of the unmanned aerial vehicle for routing inspection of the wind driven generator based on vision, the obtained suspension point of the unmanned aerial vehicle can be used for realizing the method for confirming the suspension point of the unmanned aerial vehicle for the wind driven generator, which is different from the method for confirming the suspension point of the unmanned aerial vehicle for the wind driven generator in the prior art, and is a scheme which is only based on the visual characteristics of pictures and can confirm the front and the back of the wind driven generator.

In the above solution, it can be understood that the center line of the hub of the wind turbine extends along the front or the rear. In step S1, the position coordinate may adopt an installation position of the wind turbine, the height of the hub center line is also a fixed value, and is generally recorded by a power plant, and regarding the radius planning of the circular route, a person skilled in the art can set according to specific needs, and set that the circular route is located outside the rotating track of the blades of the wind turbine, so as to enable the implementation of the method to be completed in the working process of the wind turbine, for example, in a shutdown state of the wind turbine, the circular route may be further reduced to intersect with the rotating track, and in a case that there are not infinite shooting points on the circular route, increasing the radius r of the circular route is beneficial to improving the accuracy of confirming the position of the shooting point related to the orientation identification, and decreasing the radius r of the circular route is beneficial to improving the efficiency of confirming the position of the shooting point; meanwhile, when the wind driven generator operates, the central line of the hub is generally considered to be a horizontal line, and only the yaw angle of the wind driven generator is adjusted according to the requirement, so that the annular route can be uniquely planned after the r is confirmed. In step S2, the unmanned aerial vehicle carries the aerial photography instrument to take multiple-point shots of the wind turbine along the circular route, which can be understood that the images formed by different shooting points have different shooting azimuths, so that the blades have different imaging effects in the images, and the whole position of each blade in the photo imaging area of each shooting point can be understood as that all areas of each blade which can be shot in the current shooting are located in the photo imaging area, so as to perform the length acquisition in the blade visual state in step S3. In step S3, according to the photo obtained in step S2, different blades are distributed at different positions in the circumferential direction of the hub, so that at each shooting point, the same blade has different visual lengths, and at the same shooting point, different blades have different visual lengths, and the length in the specific visual state length can be determined according to the number of pixel points between two ends of the blade. The determination of the difference between the front and rear can be made by simple image recognition by the apparent difference in the outer shapes of the hub and the nacelle. And then, setting the hovering shooting point for inspecting the wind driven generator to be positioned right above the center line of the hub of the wind driven generator on the basis of the confirmed position right in front of and/or right behind the wind driven generator.

In summary, in the above identified method for confirming the suspension point of the unmanned aerial vehicle for inspecting the wind driven generator, in order to achieve the purpose of confirming the front and the rear, the inherent or directly acquired data of the wind driven generator is utilized, and the judgment of the visual length of the blade image is further assisted.

The above set circular route is used to realize the method without stopping the wind power generator.

The unmanned aerial vehicle point of suspension when patrolling and examining that confirms can make the aerial photography appearance in the method process of patrolling and examining that carries on as follows providing for the blade of being shot is in the below of point of suspension all the time, adopts the shooting mode from top to bottom, and the gained photo background is ground or sea, and is higher with the blade contrast of common pure white, carries out blade image extraction and discernment more easily: if the point position is shot from bottom to top, the picture may use the sky as the background, the contrast with the blade is small, the blade image extraction is not easy to be carried out, and dazzling and overexposure may be generated due to the direct incidence of the sun, which is not favorable for the definition of the features in the image.

As a person skilled in the art, in order to confirm the position right in front of and/or right behind the wind turbine generator, any one of the following methods may be used: confirming the shooting point right in front or right behind through the shooting point and the center of the annular route; after the shooting point right in front or right behind is confirmed, confirming through the orientation of the shooting point; after the shooting points in the front and the back are confirmed, the connection line of the two shooting points is used for confirmation; after the blade root of the scheme is confirmed, for the wind driven generator with three blades, the center of the hub and the center of the engine room are confirmed by using three blade root coordinate data, and the confirmation is carried out through a connecting line of the center of the hub and the center of the engine room.

The further technical scheme of the method for confirming the suspension point of the unmanned aerial vehicle for routing inspection of the wind driven generator based on vision is as follows:

as described above, the above difference depends on the distribution of the shot points on the circular route, and the shot points cannot be distributed on the circular route infinitely, and when the shot point with the smallest difference in the obtained picture deviates from the straight front and the straight back beyond the set range, the straight front, the straight back, the hub center line or the current yaw angle confirmation of the wind turbine exceeds the allowable deviation. However, when the allowable deviation is directly set by only the correlation difference, the versatility of the allowable deviation is not provided for wind power generators having different blade lengths: to facilitate the accuracy of the determination of the direction, the hub center line or the yaw angle, a wind turbine with longer blade length may suitably increase the difference, and a wind turbine with shorter blade length may suitably decrease the difference, further arranged to: in step S3, a difference optimization step is further included, where the difference optimization step is:

and dividing the minimum difference value by the maximum blade length or the minimum blade length, and comparing the obtained result with a set judgment threshold value:

when the result is smaller than or equal to the judgment threshold value, confirming that the shooting point corresponding to the picture with the minimum difference value is positioned at the shooting point right in front of or right behind the wind driven generator;

and when the result is larger than the judgment threshold, adjusting the distribution of the shooting points in the step S2 to obtain a new result, comparing the adjusted result with the judgment threshold until the distribution of the shooting points is adjusted to be that the result is smaller than or equal to the judgment threshold, and then confirming that the shooting point corresponding to the picture with the minimum difference value is positioned at the shooting point right in front of or right behind the wind driven generator. According to the scheme, the result is further obtained after the minimum difference value is obtained, then the result is compared with the judgment threshold value, whether the current minimum difference value is within the acceptable range or not is confirmed, when the minimum difference value is acceptable, the shooting point associated with the picture is associated with the current minimum difference value, the straight front or the straight back of the wind driven generator is confirmed, when the minimum difference value is not acceptable, the shooting point on the annular route is adjusted, and the straight front and the straight back of the wind driven generator are confirmed again according to the corresponding comparison method. This step is used by the skilled person to improve the accuracy of the orientation verification by means of a difference check and adjustment and to adapt the difference optimization step to different sizes of wind turbines, which is not a necessary step in every embodiment according to the required accuracy of the orientation verification of the wind turbines, according to the way the shoot points on the circular route are planned.

The length of the blade in the visual state is obtained by identifying the relative positions of two ends of the blade in a photo, and specifically comprises the following steps:

s3-1, identifying the leaves in the picture and adding a target boundary frame to each leaf;

s3-2, confirming the blade root of each blade, specifically:

establishing a minimum circumscribed rectangle associated with all midpoints according to the midpoints of the target bounding boxes;

judging the number of vertexes of each target bounding box in the minimum circumscribed rectangle, if only one vertex of the target bounding box is located in the minimum circumscribed rectangle, the vertex located in the minimum circumscribed rectangle is the blade root of the blade corresponding to the target bounding box;

if the target bounding box has two vertexes positioned in the minimum circumscribed rectangle, respectively taking each vertex of the two vertexes and other target bounding boxes as vertexes of the blade root to form a polygon, and taking the vertex of the polygon with the minimum area as the blade root of the blade corresponding to the target bounding box;

and S3-3, obtaining the length of each blade in each photo in the visual state according to the confirmed blade root and blade tip of each blade. In the prior art, the target bounding box is a rectangular frame, the shape of the blade is generally determined by taking the blade tip and the blade root of the blade as different vertexes of the target bounding box respectively, the midpoint of each target bounding box is the intersection point of the connecting lines of the diagonals of each target bounding box, meanwhile, the conventional wind driven generator is generally provided with three blades, and the three blades are generally positioned at the shooting point right in front of or right behind the wind driven generator on the annular route or under the shooting point closest to the right in front of or right behind the wind driven generator: the target boundary frame of the two blades has one vertex positioned in the minimum external rectangle, the target boundary frame of the other blade has two vertices positioned in the minimum external rectangle, the vertex of the target boundary frame as the blade root can be confirmed through the method, the vertex of the target boundary frame as the blade tip can be clearly confirmed, the distance from the vertex of the target boundary frame as the blade root to the vertex of the target boundary frame as the blade tip is determined as the length of the blade, the two vertices are generally a pair of diagonal vertices of the target boundary frame, and the polygon is generally a triangle. As a person skilled in the art can understand, when the shooting point on the circular route deviates from the straight front or straight back greatly, because part of the positions of part of the blades are shielded by the hub and the nacelle, the blade root in the above step S2 is the end of the camera where the blade is far away from the blade tip in the current visual state, when the blade is completely shielded, the length of the corresponding blade can be considered as zero, when the camera moves to the extension line of a single blade for shooting, the length of the camera in the current visual state of all the blades can be considered as unavailable, and then the shooting point can be directly judged not to be located in the straight front or straight back by lacking the effective length data of all the blades as the judgment standard. In particular, for the case where the pattern of the nacelle tail may be irregular, the method can be used not only to determine the position of the midpoint of the hub, but also to determine the position of the midpoint of the nacelle end, specifically: and confirming the midpoint position of the end part of the cabin/the midpoint position of the hub of the wind driven generator by using the blade root of each blade determined in the picture with the minimum difference. When the method is used for judging the center position of the hub, compared with the method that the hub image outline is directly identified and the center point of the hub image outline is used as the hub center, the obtained hub center is more accurate because the size of the obtained image of the target boundary frame is larger than that of the hub. Regarding the determination of the midpoint position, the center position can be obtained from the position of the blade root through simple data operation by utilizing the characteristic that the blade root is generally distributed on the circumference taking the midpoint as the circle center.

For the convenience of automatic execution of the scheme, the data record in the scheme is set as follows: and establishing a fluctuation curve in the difference optimization step, wherein the fluctuation curve is used for reflecting the variation trend of the result of each photo under the annular route, filtering the fluctuation curve to obtain a correction curve, comparing the correction curve with a set judgment threshold, confirming a shooting point right in front of or right behind the wind driven generator or reacquiring the fluctuation curve so as to finally obtain the shooting point right in front of or right behind the wind driven generator. Compared with direct table recording of data, the method for recording data by using the fluctuation curve and filtering to obtain the correction curve has the characteristics of large data bearing capacity and visual result. More specifically, the above fluctuation curves may be set in a table, the vertical direction of the table corresponds to the result of each picture, and the horizontal direction of the table corresponds to the shooting time or azimuth data.

The scheme also discloses a wind driven generator inspection method based on vision, wherein on the unmanned aerial vehicle suspension point confirmed by the suspension point confirmation method, a aerial camera with a holder is used for collecting blade image data of the wind driven generator;

meanwhile, the specific image data acquisition process is as follows:

on the suspension point, changing the direction of a camera lens through a holder to adjust a field area;

the orientation changing mode is as follows: on a rotation plane of the target blade, the field of view region is translated along a line parallel to the rotation plane;

when the target blade rotates to the field of view area, the camera collects an image;

the images collected at the hovering point position comprise a first image and a second image, on the rotating track of the target blade, the first image is an image collected by the camera when the target blade is at the first position, the second image is an image collected by the camera when the target blade is at the second position, and on the rotating plane, the target blade at the first position rotates 180 degrees to reach the second position.

In the prior art, blades of wind power generators are generally installed to have a certain elevation angle and a certain cone angle, and due to the existence of the cone angle, the rotation plane can be regarded as a plane perpendicular to the rotation axis of the blade and a plane on which a circular ring is formed by rotating any point on the blade. When the lens realizes that the field of view area translates along a straight line under the action of the holder, when the rotating angle of the lens is large enough, the lens can form a strip-shaped shooting area on a rotating plane, so that the target blade can enter the field of view area at two positions, wherein one position is a first position, the other position is a second position, and the target blade at the first position reaches the second position after rotating 180 degrees. In order to realize the translation of the field area along a straight line, the lens rotates around a single shaft on the holder, and the shaft is perpendicular to the straight line. Meanwhile, the target blade can be understood as a blade needing to be inspected in three blades, and can be a part of blades on the wind driven generator or all the blades.

Be different from prior art, this scheme provides the technical scheme that the image of shooing includes first image and second image on unmanned aerial vehicle hovers, the camera lens orientation passes through the cloud platform adjustment, adopts this scheme, not only can realize that the blade does not stall and patrols and examines, from coverage area and image feature recognition angle simultaneously, has the simple characteristics of route planning.

Specifically, the scheme utilizes: at the hovering point, a first image and a second image can be obtained by changing the orientation of a camera lens through the tripod head, and after the target blade rotates 180 degrees relative to the target blade at the first position, for the surface of the target blade exposed in the field of view, when the unmanned aerial vehicle hovering point is located at the front upper side of the blade, the first image obtained at the first position can clearly show details of the lower top surface (front edge) and the front upper side (windward side upper side) of the target blade, the second image obtained at the second position can clearly show details of the bottom surface (rear edge relative to the top surface) and the front lower side (windward side lower side relative to the upper side) of the target blade; when the target blade is shot at other relative positions, even if the same target area on the target blade appears in different view field areas, the first image and the second image are obtained, namely, the shooting angle of the target area is changed, the corresponding shielding/clear area is changed in imaging, the surface defects of the blade are reflected in the image in different modes, particularly, the oblique shooting mode is easier to obtain the defect details which are obviously compared with the surrounding area aiming at the wind erosion defects which easily appear on the blade, and therefore, more blade surface image characteristic information is favorably obtained, and the blade surface defect identification rate and the defect definition degree are improved.

In conclusion, the scheme can obtain the image information of different positions of the target blade at the same hovering point, can extract the surface characteristic information of the blade at different shooting angles, and can obtain the surface characteristic information of the target blade relatively comprehensively and clearly by simple visual field area adjustment compared with the prior art that an unmanned aerial vehicle flies and shoots around the target blade in a spiral manner when a wind driven generator stops and patrols, the unmanned aerial vehicle tracks and shoots in the rotating process of the wind driven generator along with rotation tracking, a cloud deck rotates and changes a shooting area to cover blade paragraphs at different radial positions in the rotating process of the wind driven generator, and the surface characteristic information of the target blade can be obtained comprehensively on the premise of setting fewer hovering points and shorter shooting routes to achieve the patrolling purpose, so that the purpose of conveniently planning routes is achieved.

As a person skilled in the art, in the conventional inspection of the existing wind turbine, an image acquisition form that covers the entire surface of the blade as much as possible is generally adopted, so in order to ensure the image feature extraction effect, for any target blade, the entire surface of the target blade is divided into a plurality of regions, and a single shooting is only performed for one of the regions, so that the preferred method is that, from the image quality perspective, on a single hovering point, for the target blade at a first position and a second position, a camera performs multiple shooting on the target blade, each shooting covers a different target blade surface, the regions are generally arranged in sequence along the length direction of the blade according to the form of the blade, and according to the later-use method of the images: if the shot pictures are combined into a complete wind driven generator blade in a splicing mode, a single picture is selected for viewing, and the like, the relative positions of two adjacent areas can be specified. The difference between the above proposed scheme and the prior art is that the image acquired by the hovering point location includes a first image and a second image, and the shooting requirements of the first image and the second image are met through the field-of-view region transformation, so it should be understood that: at the hovering point, no matter how many times of image acquisition is performed on the target blade at the first position or the second position, the image acquisition should fall within the scope of the concept of the scheme.

The suspension points of the unmanned aerial vehicle during inspection are confirmed on the basis of confirming the front and the back, and besides the outstanding image background rejection and image quality effects, the suspension points are shot in front or at the back of the wind driven generator no matter aiming at the first image or the second image, and under the conditions that the number of the suspension points is as small as possible and the surface of the blade is covered as comprehensively as possible, the blade in the obtained picture can be shielded by the hub and the engine room of the wind driven generator as far as possible, so that the comprehensiveness of data acquisition during inspection is facilitated.

The visual inspection method for the wind driven generator is further characterized in that:

the method can fully utilize all fields of view of the hovering point position to acquire images of all acquirable areas of the target blade under the hovering point position, so that route planning is simplified and inspection efficiency is improved; meanwhile, a plurality of images are adopted to form the first image and the second image, so that a single image can more clearly reflect the details of the surface of the blade; meanwhile, the technical scheme for guaranteeing the image acquisition efficiency is set as follows through stipulating the acquisition sequence of the first image and the second image: the first image and the second image both comprise a plurality of images;

in all the images collected by the hovering point location camera:

all the collectable areas on the first position target blade are covered by the first image;

the whole collectable area on the second position target blade is covered by the second image;

in the shooting process, when all the first images are collected, collecting a second image;

the first image and the second image are acquired in the following modes: the holder enables the field of view area to move along the radial direction of the target blade, and the images are sequentially collected from one end of the target blade to the other end of the target blade.

When the scheme is implemented specifically, according to the long-strip-shaped structural characteristics of the blade, the multiple images respectively included in the first image and the second image can be images corresponding to multiple areas which are sequentially arranged along the length direction of the blade; all images acquired by the hovering point camera cover all the acquirable areas, namely the view field range of the aerial camera in the current hovering state is utilized as much as possible; the second image acquisition is carried out after the first image acquisition is limited, namely the single rotation angle of the holder is smaller on the basis of meeting the shooting setting, so that the image acquisition time can be shortened; the first image and the second image acquisition modes defined above are used for defining the image acquisition sequence for matching the problem of shortening the acquisition time as set forth above. In one particular implementation, image acquisition is accomplished as follows: after the aerial photography instrument hovers along with the unmanned aerial vehicle, image acquisition is carried out on a target aerial photograph at a first position, the direction of the tripod head is adjusted by driving the lens to move in a mode that the view field area moves along the radial direction of the target blade at the first position, the first image acquisition sequence is from the blade root to the blade tip and from the blade tip to the blade root, after the first image acquisition is completed, the tripod head drives the lens to be adjusted to the direction capable of acquiring the target blade at a second position, and then all second image acquisition is completed according to the sequence from the blade root to the blade tip or from the blade tip to the blade root. As a person skilled in the art, when the target blade is used to have a predetermined inspection range, the images included in the first image and the second image may cover the predetermined inspection range.

For through later stage image concatenation, obtain the integrated image or the model that can each position surface condition of visual reflection target blade, set up to: images corresponding to two adjacent acquisition areas in the first image have a set overlapping rate;

the images corresponding to two adjacent acquisition regions in the second image have a set overlapping rate. It can be understood that the above overlapping rate is used for stitching between related images, a specific stitching mode may adopt a conventional feature recognition mode, and the specific overlapping rate requirement is set according to the performance of the camera, the requirements of the employed stitching software, and the like.

When the wind driven generator operates, the front edge of the blade is used as a wind cutting surface for wind cutting measurement, the front edge of the blade and the windward side of the blade are generally regions with serious wind erosion, and in order to more clearly reflect the image characteristics of the regions, the wind driven generator is set as follows: the hovering position of the unmanned aerial vehicle or the position of the target blade when the image is acquired satisfies the following conditions: when the camera collects images, the image target area on the target blade is positioned at the lower side of the lens. The scheme is as follows: when the camera collects images, the image collection is completed above an image collection area, if the position of the camera is the front upper side of the blade, the area with good blade surface feature recognition quality at the first position is the front edge and the windward upper side, the area with good blade surface feature recognition quality at the second position is the rear edge and the windward lower side, when the position of the camera is adjusted to the back upper side of the blade, the area with good blade surface feature recognition quality at the first position is the front edge and the leeward upper side, and the area with good blade surface feature recognition quality at the second position is the rear edge and the leeward lower side, so that a comprehensive blade surface image can be obtained through two points. In addition, a shooting mode from top to bottom is adopted, the background of the obtained picture is the ground or sea surface, the contrast ratio of the picture to common pure white blades is higher, and the blade image extraction and identification are easier to perform: if the point position is shot from bottom to top, the picture may use the sky as the background, the contrast with the blade is small, the blade image is not easy to extract, and dazzling light and overexposure may be generated due to direct incidence of the sun, which is not favorable for definition of features in the image. In specific implementation, for the first position and the second position proposed above, and the first position and the second position both include the application of multiple images, it is preferable to adopt: the target blade and the ground enter a view field area and are shot when keeping the level, so that more pixel points are occupied by blade ridges in each picture, the visual approximation degree of the adjacent area of the blade is high, the background image is convenient to identify and reject, the blade has the characteristic of high image characteristic consistency, and when the three-dimensional blade surface image modeling is carried out in the later stage, the quality of a mapping can be effectively guaranteed so as to be beneficial to judging the surface defects of the blade, namely: the first image and the second image are both images acquired when the target blade rotates to a horizontal posture. In order to obtain comprehensive blade inspection data, the suspension points comprise a first suspension point located on the front side of the central line of the hub of the wind driven generator and a second suspension point located on the rear side of the central line of the hub of the wind driven generator.

The invention has the following beneficial effects:

in the method for confirming the suspension point of the unmanned aerial vehicle for inspecting the wind driven generator, in order to achieve the aim of confirming the dead ahead and the dead behind, the inherent or directly acquired data of the wind driven generator is utilized, the judgment of the visual length of the blade image is further assisted, the whole method can be completed by adopting the existing aerial photography instrument, in order to achieve intelligent automatic inspection, the suspension point confirming method can be completed by simple image characteristic identification, through a pure visual load scheme of the unmanned aerial vehicle, the dead ahead and/or the dead behind of the wind driven generator is determined under the condition that the wind driven generator does not shut down, and the current yaw angle of the wind driven generator is analyzed.

The unmanned aerial vehicle suspension point during patrolling and examining confirmed can make the aerial photography appearance in carrying out the method process of patrolling and examining that provides as follows for the blade of being shot is in the below of suspension point all the time, adopts the shooting mode from the top down, and the gained photo background is ground or sea, and is higher with the blade contrast of common pure white, carries out blade image extraction and discernment more easily: if the point position is shot from bottom to top, the picture may use the sky as the background, the contrast with the blade is small, the blade image extraction is not easy to be carried out, and dazzling and overexposure may be generated due to the direct incidence of the sun, which is not favorable for the definition of the features in the image.

Drawings

Fig. 1 is a schematic diagram of a suspended point of an unmanned aerial vehicle relative to a wind driven generator during inspection according to a specific application embodiment of the method for confirming the suspended point of the unmanned aerial vehicle during inspection of the wind driven generator based on vision;

fig. 2 is a schematic diagram of a shooting flow reflecting a suspension point right in front of a wind driven generator in a specific application embodiment of the vision-based wind driven generator inspection method;

fig. 3 is a schematic diagram of a shooting flow reflecting a suspension point right behind a wind driven generator in a specific application embodiment of the vision-based wind driven generator inspection method;

FIG. 4 is a schematic view of the coverage area of the target blade in the shooting process shown in FIG. 2 and FIG. 3;

fig. 5 is a schematic diagram showing a region of each of a first image and a second image on a blade in a specific application embodiment of the vision-based wind turbine inspection method according to the present disclosure;

FIG. 6 is a schematic diagram of the division of the appearance region of a blade of a conventional wind turbine;

fig. 7 is a schematic diagram for explaining related technical terms in an implementation manner of the unmanned aerial vehicle suspension point determination method provided in embodiment 11;

FIG. 8 is a diagram showing a wave curve in one embodiment;

FIG. 9 is used to show a correction curve in one embodiment.

The reference numerals in fig. 6 are respectively: 1. trailing edge, 2, leading edge, 3, windward side, 4, blade tip.

Detailed Description

The present invention will be described in further detail with reference to examples, but the present invention is not limited to the following examples:

example 1:

as shown in fig. 1 to 6, in the method for confirming the suspension point of the unmanned aerial vehicle for the inspection of the wind driven generator based on the vision, the suspension point of the unmanned aerial vehicle during the inspection is confirmed to be positioned right above the central line of the hub of the wind driven generator by confirming the right front and/or the right back of the wind driven generator;

wherein, the method for confirming the right front and/or the right back comprises the following steps:

s1, confirming an annular air route of the unmanned aerial vehicle according to the position coordinate of the wind driven generator and the height of the central line of the hub: the annular route is positioned on the outer side of the rotating track of the blades of the wind driven generator, the annular route is positioned on the horizontal plane at the height, and the position coordinate is positioned on the central line of the annular route;

s2, shooting the blades of the wind driven generator by the unmanned aerial vehicle in the flying process along the annular air route, wherein the annular air route is provided with a plurality of shooting points, the lens of each shooting point faces the center of the annular air route, and the blades are integrally positioned in the picture imaging area of each shooting point;

s3, obtaining the length of each blade in each photo in the visual state, and determining the right front or right back of the wind driven generator according to the difference value of the maximum blade length and the minimum blade length aiming at each photo;

Example 2:

the present embodiment is further refined on the basis of embodiment 1:

in step S3, a difference optimization step is further included, where the difference optimization step is:

and when the result is larger than the judgment threshold, adjusting the distribution of the shooting points in the step S2 to obtain a new result, comparing the adjusted result with the judgment threshold until the distribution of the shooting points is adjusted to be that the result is smaller than or equal to the judgment threshold, and then confirming that the shooting point corresponding to the picture with the minimum difference value is positioned at the shooting point right in front of or right behind the wind driven generator.

Example 3:

this example is further refined on the basis of example 1:

s3-1, identifying the leaves in the picture and adding a target bounding box to each leaf;

s3-2, confirming the blade root of each blade, specifically:

if the target boundary frame has two vertexes positioned in the minimum external rectangle, respectively taking each vertex of the two vertexes and other target boundary frames as vertexes of the blade root to form a polygon, and taking the vertex of the polygon with the minimum area as the blade root of the blade corresponding to the target boundary frame;

and S3-3, obtaining the length of each blade in each photo in the visual state according to the confirmed blade root and blade tip 4 of each blade.

Example 4:

this example is further refined on the basis of example 3:

and confirming the midpoint position of the end part of the cabin/the midpoint position of the hub of the wind driven generator by using the blade root of each blade determined in the picture with the minimum difference.

Example 5:

the embodiment is further refined on the basis of the embodiment 2:

and establishing a fluctuation curve in the difference optimization step, wherein the fluctuation curve is used for reflecting the variation trend of the result of each photo under the annular route, filtering the fluctuation curve to obtain a correction curve, comparing the correction curve with a set judgment threshold, confirming a shooting point right in front of or right behind the wind driven generator or reacquiring the fluctuation curve so as to finally obtain the shooting point right in front of or right behind the wind driven generator.

Example 6:

the embodiment provides a wind driven generator inspection method based on vision on the basis of the embodiment 1, and the method comprises the following steps: collecting image data of a blade of a wind driven generator by using a aerial camera with a holder on the suspension point of the unmanned aerial vehicle confirmed by the suspension point confirming method in any one of embodiments 1 to 5;

meanwhile, the specific image data acquisition process is as follows:

Example 7:

this example is further refined on the basis of example 6:

the first image and the second image both comprise a plurality of images;

in all images acquired by the hovering point location camera:

all the acquirable areas on the second position target blade are covered by the second image;

in the shooting process, when all the first images are acquired, acquiring a second image;

the first image and the second image are acquired in the following modes: the holder enables the field of view area to move along the radial direction of the target blade, and each image is sequentially collected from one end to the other end of the target blade. When the wind driven generator is in operation, the wind driven generator blade can be divided into four parts from the appearance: the PS plane (windward side 3), the SS plane (leeward side, on the back side of the windward side 3), the leading edge 2 (wind entry side), and the trailing edge 1 (wind exit side), as shown in fig. 6. When the blade works, the front edge 2 of the blade is used as a wind cutting surface for wind cutting-in measurement, the front edge 2 and the windward side 3 of the blade are generally severe wind erosion areas, and in order to more clearly reflect the image characteristics of the areas, the front edge 2 and the windward side 3 are set as follows: the hovering position of the blade unmanned aerial vehicle meets the following conditions: when the camera collects images, the target blade is positioned at the lower side of the lens. The scheme is as follows: when the camera collects images, the image collection is completed from the upper part of an image collection area, if the position of the camera is the front upper part of the blade, the area with good blade surface feature recognition quality at the first position is the front edge 2 and the upper side of the windward side 3, the area with good blade surface feature recognition quality at the second position is the rear edge 1 and the lower side of the windward side 3, when the position of the camera is adjusted to the back upper part of the blade, the area with good blade surface feature recognition quality at the first position is the front edge 2 and the upper side of the leeward side, and the area with good blade surface feature recognition quality at the second position is the rear edge 1 and the windward side 3 of the leeward side 3, so that a comprehensive blade surface image can be obtained through two points. In addition, by adopting a shooting mode from top to bottom, the background of the obtained picture is the ground or sea surface, the contrast ratio of the picture to the common pure white blade is higher, and the blade image is easier to extract and identify: if the point position is shot from bottom to top, the picture may use the sky as the background, the contrast with the blade is small, the blade image extraction is not easy to be carried out, and dazzling and overexposure may be generated due to the direct incidence of the sun, which is not favorable for the definition of the features in the image. In specific implementation, for the first position and the second position proposed above, and the first position and the second position both include the application of multiple images, it is preferable to adopt: the target blade enters a field of view area and is shot when the target blade and the ground are kept horizontal, so that more pixel points are occupied by blade ridges in each picture, the visual approximation degree of the adjacent area of the blade is high, the background image can be identified and eliminated conveniently, the blade has the characteristic of high image characteristic consistency, and the quality of a mapping can be effectively guaranteed to be beneficial to judging the surface defects of the blade when the three-dimensional blade surface image modeling is carried out in the later period.

Example 8:

the present embodiment is further refined on the basis of embodiment 6:

images corresponding to two adjacent acquisition areas in the first image have a set overlapping rate;

the images corresponding to two adjacent acquisition regions in the second image have a set overlapping rate.

Example 9:

the present embodiment is further refined on the basis of embodiment 6:

the hovering position of the unmanned aerial vehicle or the position of the target blade when the image is acquired satisfies the following conditions: when the camera collects images, the image target area on the target blade is positioned at the lower side of the lens.

Example 10:

this example is further refined on the basis of example 9:

the first image and the second image are both images acquired when the target blade rotates to a horizontal posture;

the suspension point comprises a first suspension point positioned on the front side of the central line of the hub of the wind driven generator and a second suspension point positioned on the rear side of the central line of the hub of the wind driven generator.

Example 11:

the same concept as that of the above embodiments, this embodiment provides a specific implementation manner of the method for confirming the suspension point of the unmanned aerial vehicle:

the method mainly comprises the following steps:

step 1: and controlling the unmanned aerial vehicle to fly to the height H of the central line of the hub and fly around the central line of the wind driven generator in a circular route with the radius r as shown in figure 1.

Step 2: the method comprises the following steps of visually identifying the front A0 and the back B0 of the center of the wind hub in the surrounding flight process, wherein the specific steps are as follows:

step 2-1: identifying the leaves, adding the target bounding boxes to the leaves, one can get the midpoint of each target bounding box and the 4 vertices of each target bounding box, as shown by the rectangular boxes surrounding each leaf and the midpoints in these rectangular boxes in FIG. 7.

Step 2-2: the root of each blade is identified.

The method comprises the following specific steps: establishing a minimum circumscribed rectangle (hereinafter referred to as circumscribed moment) according to the midpoints of the three target bounding boxes, as shown in a rectangular frame established by the three midpoints in fig. 7, if only one vertex of the target bounding box is in the circumscribed moment, determining that the vertex of the target bounding box in the circumscribed moment is the blade root of the blade (as shown in fig. 7, the vertexes, which are positioned in the circumscribed moment, of the target bounding boxes of the upper left blade and the lower left blade are respectively the blade root of the corresponding blade); if 2 target bounding box vertices (for example, the upper right blade in fig. 7) are included in the external moment, and the vertices of the target bounding box are respectively formed into triangles with the other two target bounding boxes as the vertices of the blade root, and the vertex of the triangle with the smallest area in the two triangles is determined as the blade root of the blade (that is, the 2 vertices of the upper right blade in the target bounding box are respectively formed into triangles with the blade roots of the upper left blade and the lower left blade, and the vertex of the upper right blade target bounding box in the external moment for forming the triangle with the smallest area is determined as the blade root of the upper right blade).

In fig. 7, reference numerals are respectively: 5. target boundary frame of left upper blade, 6, target boundary frame of left lower blade, 7, target boundary frame of right upper blade, 8, external torque, 9, hub center.

In addition, as shown in fig. 7, the coordinates of the center of the hub in the figure can be obtained by averaging or the like according to the relationship between the three coordinates of the blade root (indicated by dots in fig. 7) and the center of the hub and the center of the nacelle. Similarly, if the view port is located directly behind the wind turbine, the nacelle center coordinates may be obtained.

Step 2-3: the blade length is determined.

The method comprises the following specific steps: the distance from the target bounding box vertex determined as the blade root to the target bounding box vertex determined as the blade tip 4 (typically, the distance from the target bounding box diagonal vertex) is determined as the blade length.

Step 2-4: for each shot point on the circular route, a fluctuation curve can be obtained according to (longest blade length-shortest blade length)/shortest blade length, as shown in fig. 8.

Step 2-5: the above-mentioned fluctuation curve is filtered to obtain a filtered curve, as shown in fig. 9.

Step 2-6: the wave trough is selected to be the right front or the right back of the wind driven generator, and if the wave trough is selected to be smaller than 0.08, the position is judged to be the right front or the right back. (it should be noted that, in fig. 8 and 9, different longitudinal positions are used for different ratios of the pointer, and different transverse positions are used for different shooting times of the pointer, and the valley is longer in the figures because the drone hovers at the position for a period of time, and according to the arrangement mode of different shooting point distributions on the circular route, when there are a plurality of values less than 0.08, it indicates that there are a plurality of shooting points that satisfy the setting condition and can be used to determine that the shooting point is located right in front of or right behind the wind turbine, but the shooting point corresponding to the minimum value should be selected as the shooting position located right in front of or right behind the wind turbine).

Further, after obtaining dead ahead or dead behind, can also openly have wheel hub according to aerogenerator, based on image recognition wheel hub, judge that unmanned aerial vehicle is in dead ahead.

And 3, step 3: the unmanned aerial vehicle is controlled to hover at the point A to shoot an image in front of the wind driven generator after vertically rising at the point A0 right in front by the distance h, and flies to the point B to shoot an image behind the wind driven generator after shooting is finished, wherein the point B is a point behind the vertically rising point B0 right behind by the distance h, and the vertical rising distance h can also be set manually. Or, the unmanned aerial vehicle can be controlled to shoot in the mode B before shooting in the mode A.

Example 12:

the embodiment provides a specific inspection method implementation mode:

when the unmanned aerial vehicle hovers at the point a shown in fig. 1, the shooting process shown in the left diagram and the shooting process shown in the right diagram of fig. 2 are performed, so as to obtain a first shooting process coverage area and a second shooting process coverage area shown by arrows in fig. 4; when hovering at the point B, the shooting process shown in the left diagram of FIG. 3 and the shooting process shown in the right diagram of FIG. 3 are performed, so as to obtain a third shooting process coverage area and a fourth shooting process coverage area shown by arrows in FIG. 4.

By way of example with the first shooting flow, it can be known from fig. 2 that blade image acquisition is divided into a plurality of acquisition sections along the length direction of the blade, and on the blade, the acquisition sections 2 are sequentially arranged in the direction away from the center of the fan and close to the center of the fan as the acquisition section 1, and so on to the acquisition section n. After the wind driven generator with the three blades rotates for one circle, after the information of 3 blades is collected in the field of view corresponding to the collection section 1 during collection, the pan-tilt control camera rotates and focuses to the field of view corresponding to the collection section 2, the information of 3 blades passing through the field of view is collected until the information of 3 blades under the area corresponding to the collection section n is collected, and then the pan-tilt control camera rotates to sample a second shooting flow. And then, the unmanned aerial vehicle flies to the point B to collect the shot data in the same way until the whole shooting process is collected.

Example 13:

on the basis of embodiment 12, the present embodiment provides a specific inspection method implementation manner:

the specific inspection process is as follows:

step 1: the leaf is divided into n acquisition segments, a, B, C, D, E, etc., as shown in fig. 5. Adjacent acquisition segments may partially overlap. Ensuring that each part of the blade is acquired and has a corresponding image overlap ratio.

Step 2: fly to point location A shown in fig. 1 through manual control or unmanned aerial vehicle is automatic to hover and be located point location A, aim at the A position of blade with unmanned aerial vehicle camera lens through the cloud platform and keep motionless, when the blade is rotatory when the A position, the camera is taken a candid photograph. The fan blade rotates a circle, and the camera can sequentially capture image information of the A positions of the blade 1, the blade 2 and the blade 3. The unmanned aerial vehicle camera has the automatic function of focusing fast, when the blade rotated to A position, automatic fast focusing and accomplished image acquisition.

And step 3: after the A positions of the three blades are collected, the camera lens is aligned to the B positions of the blades through the rotation of the cradle head, image collection is conducted on the B positions of the three blades, 8230, the process is repeated until the image information of the three blades is collected at the point position A, the images are first images, and then second images of the three blades are collected at the point position A.

And 4, step 4: the method comprises the steps of flying to a point B shown in the figure 1 through manual control or automatic unmanned aerial vehicle, hovering the point B, repeating the step 2 and the step 3, collecting image information of three blades when the point B is collected, wherein the images comprise a first image and a second image, and collecting the second image after the first image of the point B is collected.

And 5, synthesizing a longer complete fan blade photo according to the acquired image information.

The foregoing is a more detailed description of the present invention in connection with specific preferred embodiments thereof, and it is not intended that the specific embodiments of the present invention be limited to these descriptions. For those skilled in the art to which the invention pertains, other embodiments that do not depart from the gist of the invention are intended to be within the scope of the invention.

Claims

1. The method for confirming the suspension point of the unmanned aerial vehicle for the wind driven generator inspection based on vision is characterized in that the suspension point of the unmanned aerial vehicle during inspection is confirmed to be positioned right above the central line of the hub of the wind driven generator by confirming the right front and/or the right back of the wind driven generator;

s1, confirming an annular air route of unmanned aerial vehicle flight according to the position coordinate of the wind driven generator and the height of the central line of the hub: the annular route is positioned on the outer side of the rotating track of the blades of the wind driven generator, the annular route is positioned on the horizontal plane at the height, and the position coordinate is positioned on the central line of the annular route;

s2, shooting the blades of the wind driven generator by the unmanned aerial vehicle in the flying process along the annular air route, wherein the annular air route is provided with a plurality of shooting points, and the blades are integrally positioned in the photo imaging area of each shooting point;

2. The vision-based wind turbine inspection unmanned aerial vehicle suspension point confirmation method according to claim 1,

3. The vision-based method for confirming the suspension point of the unmanned aerial vehicle for routing inspection of the wind driven generator according to claim 1, wherein the length of the blade in the visual state is obtained by identifying the relative positions of two ends of the blade in a photo, and specifically comprises the following steps:

s3-2, confirming the blade root of each blade, specifically:

and S3-3, obtaining the length of each blade in each photo in the visual state according to the confirmed blade root and blade tip of each blade.

4. The vision-based method for confirming the suspension point of the unmanned aerial vehicle for routing inspection of the wind driven generator is characterized in that each shooting point lens faces to the center of the circular route.

5. The vision-based unmanned aerial vehicle suspension point confirmation method for inspecting the wind driven generator by the wind power generator is characterized in that a fluctuation curve is established in the difference optimization step, the fluctuation curve is used for reflecting the change trend of the result of each photo under the annular air route, the fluctuation curve is filtered to obtain a correction curve, the correction curve is compared with a set judgment threshold, a shooting point right in front of or right behind the wind driven generator is confirmed, or the fluctuation curve is obtained again, so that a shooting point right in front of or right behind the wind driven generator is finally obtained.

6. The visual inspection method for the wind driven generator is characterized in that an unmanned aerial vehicle suspension point confirmed by the suspension point confirmation method according to any one of claims 1 to 5 is used for collecting blade image data of the wind driven generator by using a aerial camera with a holder;

meanwhile, the specific image data acquisition process comprises the following steps:

7. The vision-based wind turbine inspection method according to claim 6, wherein the first image and the second image each include a plurality of images;

in all images acquired by the hovering point location camera:

8. The vision-based wind turbine inspection method according to claim 6 or 7, wherein images corresponding to two adjacent acquisition regions in the first image have a set overlapping rate;

9. The vision-based wind turbine inspection method according to claim 6, wherein the hovering position of the unmanned aerial vehicle or the position of the target blade when the image is acquired satisfies: when the camera collects images, the image target area on the target blade is positioned at the lower side of the lens.

10. The vision-based wind turbine inspection method according to claim 9, wherein the first image and the second image are both images acquired when the target blade is rotated to a horizontal posture;