CN112348034A - Crane defect detection system based on unmanned aerial vehicle image recognition and working method - Google Patents
Crane defect detection system based on unmanned aerial vehicle image recognition and working method Download PDFInfo
- Publication number
- CN112348034A CN112348034A CN202011133457.1A CN202011133457A CN112348034A CN 112348034 A CN112348034 A CN 112348034A CN 202011133457 A CN202011133457 A CN 202011133457A CN 112348034 A CN112348034 A CN 112348034A
- Authority
- CN
- China
- Prior art keywords
- image
- aerial vehicle
- unmanned aerial
- crane
- defect
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 140
- 238000001514 detection method Methods 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000011156 evaluation Methods 0.000 claims abstract description 67
- 230000000007 visual effect Effects 0.000 claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 27
- 238000004891 communication Methods 0.000 claims abstract description 23
- 238000010223 real-time analysis Methods 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims abstract description 14
- 239000013598 vector Substances 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 11
- 230000011218 segmentation Effects 0.000 claims description 9
- 239000002184 metal Substances 0.000 claims description 7
- 230000009467 reduction Effects 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000012706 support-vector machine Methods 0.000 claims description 5
- 238000000547 structure data Methods 0.000 claims description 3
- 238000011017 operating method Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 7
- 238000007689 inspection Methods 0.000 description 7
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 4
- 230000001276 controlling effect Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000005299 abrasion Methods 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 206010034719 Personality change Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000001179 sorption measurement Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C39/00—Aircraft not otherwise provided for
- B64C39/02—Aircraft not otherwise provided for characterised by special use
- B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
Abstract
The invention discloses a crane defect detection system based on unmanned aerial vehicle image recognition, which comprises an unmanned aerial vehicle body, an unmanned aerial vehicle main controller, an airborne detection device, a communication link system and a ground control console, wherein the unmanned aerial vehicle main controller is connected with the ground control console; the real-time analysis system adopts image local area constraint and deconvolution image algorithms to improve the resolution of the collected image, and then adopts a defect identification model to obtain a crane surface defect evaluation result; the unmanned aerial vehicle main control unit adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console. The method can process the acquired surface structure image in real time, quickly obtain a relatively accurate crane surface defect evaluation result, adjust the flight direction of the unmanned aerial vehicle and the shooting angle of the airborne detection device according to the visual evaluation result, optimize the flight track and the shooting parameters of the unmanned aerial vehicle, realize effective feedback of image processing data, improve the quality of the shot image and improve the detection efficiency.
Description
Technical Field
The invention relates to the technical field of crane surface defect detection, in particular to a crane defect detection system and a working method based on unmanned aerial vehicle image recognition.
Background
The large crane is an essential key device for loading and unloading operation, however, the crane is easy to generate fatigue cracks under the effect of long-term fatigue alternating load, so the crane is required to be inspected regularly, the current inspection method is a manual visual method, the inspection period is once a year, and the inspection method has the following defects: firstly, the number of parts of the crane is large, and the manual inspection consumes long time; secondly, the safety risk is increased by manual climbing; thirdly, human error is large. Therefore, the crane safety detection by using the unmanned aerial vehicle has important significance.
In recent years, the unmanned aerial vehicle industry is rapidly developed, and particularly, the unmanned aerial vehicle is widely applied to the fields of aerial photography, mapping, power inspection and the like. At the present stage, the image recognition technology is efficient and reliable, the image recognition technology is popularized and utilized in the field of structural safety detection, the expenditure of inspection personnel is greatly saved, and the reality and the objectivity of a detection result are ensured. For example, an unmanned aerial vehicle device for detecting surface defects of a crane is disclosed in application No. CN2068045544U, and an adsorption crane surface crack defect detecting device is disclosed in application No. CN211139484U, which effectively solves many problems caused by the aforementioned artificial detection by applying an unmanned aerial vehicle and an image recognition technology to crane defect detection.
However, the crane is large in size, complex in surface structure and different in defect appearance and position, even if the routing inspection route and the shooting route of the unmanned aerial vehicle are planned in advance, the image quality is poor due to the problems of shooting positions or shooting parameters and the like, and the final defect detection recognition rate and accuracy are affected.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a crane defect detection system and a working method based on unmanned aerial vehicle image recognition, which can process the acquired surface structure image in real time, quickly obtain a relatively accurate crane surface defect evaluation result, obtain the visual evaluation result of the acquired image by combining with the image resolution ratio, and feed the visual evaluation result back to an unmanned aerial vehicle main controller, so that the unmanned aerial vehicle main controller can adjust the flight direction of the unmanned aerial vehicle and the shooting angle of an airborne detection device according to the visual evaluation result, optimize the flight track and the shooting parameters of the unmanned aerial vehicle, realize the effective feedback of image processing data, improve the quality of the shot image and improve the detection efficiency.
In order to achieve the purpose, the invention adopts the following technical scheme:
a crane defect detection system based on unmanned aerial vehicle image recognition comprises an unmanned aerial vehicle body, an unmanned aerial vehicle main controller, an airborne detection device, a communication link system and a ground control console;
the unmanned aerial vehicle main controller is used for controlling the flight track and the flight attitude of the unmanned aerial vehicle body;
the airborne detection device is mounted on the unmanned aerial vehicle body and used for collecting images on the surface of the crane and sending collected data to the unmanned aerial vehicle main controller; the unmanned aerial vehicle main controller also sends the acquired data to a ground console through a communication link system, a real-time analysis system installed in the ground console improves the resolution of the acquired image by adopting image local area constraint and deconvolution image algorithms, a defect identification model based on a fractal threshold method is adopted to process the acquired data to obtain a crane surface defect evaluation result, and the real-time analysis system adjusts the fractal threshold of the defect identification model according to the crane surface defect evaluation accuracy;
the real-time analysis system is also used for obtaining the visual evaluation result of the collected image by combining the adjusted image resolution and the crane surface defect evaluation result, sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system, and the unmanned aerial vehicle main controller adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console.
In order to optimize the technical scheme, the specific measures adopted further comprise:
further, the defect identification result comprises the position of the surface defect relative to the crane, the defect type, the range ratio of the defect area in the image and the position of the defect area in the image.
Furthermore, the airborne detection device comprises a visible light shooting device for collecting visible light images of the surface structure of the crane, a laser scanning device for collecting size data of the surface structure of the crane and an infrared light shooting device for collecting temperature difference data of the surface of the crane.
Further, airborne detection device installs on rotatory cloud platform, shoots the angle through rotatory cloud platform adjustment, and wherein, the angle of pitch scope of vertical motion direction is 90, and horizontal direction position motion range is 150.
Based on the detection system, the invention also provides a working method of the crane defect detection system based on unmanned aerial vehicle image recognition, and the working method comprises the following steps:
s1, acquiring the structural morphology of the crane to be detected, and planning the flight path of the unmanned aerial vehicle; positioning the unmanned aerial vehicle according to the position of the crane to be detected, and driving the unmanned aerial vehicle to fly along a planned path;
s2, in the flying process of the unmanned aerial vehicle, an airborne detection device is adopted to collect surface structure data of the crane, the collected structure is sent to a ground console, a real-time analysis system installed in the ground console improves the resolution of the collected image by adopting image local area constraint and deconvolution image algorithms, and then a defect identification model based on a fractal threshold method is adopted to process the collected data to obtain a crane surface defect evaluation result;
s3, obtaining a visual evaluation result of the collected image by combining the adjusted image resolution and the crane surface defect evaluation result, and sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system;
s4, the unmanned aerial vehicle main controller adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console;
and S5, adjusting the fractal threshold of the defect identification model by the real-time analysis system according to the evaluation accuracy of the crane surface defects.
Further, in step S2, the process of processing the collected data by using the defect identification model based on the fractal threshold method to obtain the crane surface defect evaluation result includes the following steps:
s21, generating a crane metal structure image library, storing a certain amount of crane surface structure image samples marked with defect types and defect positions, and denoising the crane surface structure image samples;
s22, constructing a defect identification model based on multi-scale geometric analysis and a support vector machine, wherein the defect identification model comprises an image processing unit and a defect identification unit;
s23, based on the image processing unit, performing threshold segmentation on the denoised image sample by adopting a fractal threshold method, decomposing the segmented image sample, extracting a characteristic vector of the decomposed image sample, performing dimension reduction processing on the characteristic vector, and generating a training sample;
s24, training the defect identification unit by adopting the training sample;
s24, extracting the shot surface structure image of the crane to be detected, importing the image into a defect identification model, performing threshold segmentation on the denoised surface structure image by adopting a fractal threshold method, decomposing the segmented image, extracting image characteristic vectors, performing dimensionality reduction on the characteristic vectors, and then performing real-time identification and real-time positioning on the crane surface defects.
Further, the defect identification model comprises a defect accuracy rate evaluation unit, wherein the defect accuracy rate evaluation unit is used for evaluating the crane surface defect evaluation accuracy rate within a preset time range and sending the crane surface defect evaluation accuracy rate obtained through evaluation to the image processing unit so as to adjust the fractal threshold value of the image processing unit.
The invention has the beneficial effects that:
(1) threshold segmentation is carried out on the denoised image based on a fractal threshold method, high-precision crack features are effectively extracted and identified, the collected surface structure image can be processed in real time, and a relatively accurate crane surface defect evaluation result is rapidly obtained.
(2) And dynamically adjusting the fractal threshold according to the defect identification accuracy to ensure the image identification efficiency.
(3) And the image resolution of the unmanned aerial vehicle is improved by adopting an image local area constraint and deconvolution image algorithm, so that the subsequent image identification processing is facilitated.
(4) The visual evaluation result of the collected image is obtained by combining the defect identification result and the adjusted image resolution ratio and is fed back to the unmanned aerial vehicle main controller, so that the flight direction of the unmanned aerial vehicle and the shooting angle of the airborne detection device can be adjusted according to the visual evaluation result, the flight track and the shooting parameters of the unmanned aerial vehicle are optimized, effective feedback of image processing data is realized, the quality of the shot image is improved, and the detection efficiency is improved.
Drawings
Fig. 1 is a schematic structural diagram of a crane defect detection system based on unmanned aerial vehicle image recognition.
FIG. 2 is a flow chart of a working method of the crane defect detection system based on unmanned aerial vehicle image recognition.
FIG. 3 is a flow chart of an image analysis algorithm of the defect identification model of the present invention.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
It should be noted that the terms "upper", "lower", "left", "right", "front", "back", etc. used in the present invention are for clarity of description only, and are not intended to limit the scope of the present invention, and the relative relationship between the terms and the terms is not limited by the technical contents of the essential changes.
With reference to fig. 1, the invention provides a crane defect detection system based on unmanned aerial vehicle image recognition, which comprises an unmanned aerial vehicle body, an unmanned aerial vehicle main controller, an airborne detection device, a communication link system and a ground console.
The unmanned aerial vehicle main control unit is used for controlling the flight track and the flight attitude of the unmanned aerial vehicle body.
The airborne detection device is mounted on the unmanned aerial vehicle body and used for collecting images on the surface of the crane and sending collected data to the unmanned aerial vehicle main controller; the unmanned aerial vehicle main controller also sends the acquired data to a ground console through a communication link system, a real-time analysis system installed in the ground console improves the resolution of the acquired image by adopting image local area constraint and deconvolution image algorithms, a defect identification model based on a fractal threshold method is adopted to process the acquired data to obtain a crane surface defect evaluation result, and the real-time analysis system adjusts the fractal threshold of the defect identification model according to the crane surface defect evaluation accuracy.
The real-time analysis system is also used for obtaining the visual evaluation result of the collected image by combining the adjusted image resolution and the crane surface defect evaluation result, sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system, and the unmanned aerial vehicle main controller adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console.
The crane defect detection system based on unmanned aerial vehicle image recognition comprises an unmanned aerial vehicle platform, a communication link system and a ground control console.
The unmanned aerial vehicle platform is divided into three parts of a body structure, flight control and an airborne detection device. The flight control system comprises a main controller, a flight attitude measuring system, a motor speed regulating system, a power supply module, a positioning module, an obstacle avoidance device and the like. The airborne detection device is divided into three types of visible light, infrared and laser, and the acquisition of the visible light image, the structure size data and the surface temperature difference data of the surface of the structure is realized. Preferably, in order to realize a dead-angle-free detection range, the invention is designed as follows: the camera rotating holder drives the camera to rotate, the camera is aligned to a position to be measured of the crane to carry out multi-azimuth shooting, the pitch angle range of the vertical motion direction of the camera reaches +/-90 degrees, the azimuth motion range of the camera in the horizontal direction reaches +/-150 degrees, the camera can overlook imaging downwards and look up imaging upwards, and multi-azimuth shooting of various key positions of the large crane can be carried out without viewing field and dead angles.
The communication link system comprises an unmanned aerial vehicle communication system and a detection device communication system, the unmanned aerial vehicle communication system is mainly used for ensuring data communication between a ground measurement and control station and an aerial unmanned aerial vehicle and realizing remote control and positioning of the unmanned aerial vehicle on the ground, the detection device communication system transmits acquired data information such as images to a ground control console, the detection device communication system comprises three parts of data compression, wireless transmission and data decoding, and the real-time performance and the authenticity of transmitted data need to be ensured; the link consists of an airborne module and a ground module, and a relay module is required to be arranged at a proper position of the crane with a large detection range so as to ensure the normal operation of the communication system.
The ground control console is divided into two parts of flight control and real-time data processing, wherein the flight control part is mainly used for monitoring and controlling the flight state of the unmanned aerial vehicle, navigating the unmanned aerial vehicle, controlling the switching of the detection device and the like.
The real-time data processing part is realized by a real-time analysis system, and on one hand, image data acquired by the visual detection device is displayed in real time so as to be used for the operator to carry out initial judgment on the detection result on site; on the other hand, the image processing technology is adopted to extract and identify the defect characteristics of the obtained image data information so as to realize the discrimination and confirmation of the detected structure defects and hidden dangers and realize the timely diagnosis and troubleshooting of the structure safety condition.
In the invention, in order to realize real-time performance and effective data feedback, a real-time analysis system firstly preprocesses an image and then analyzes the image:
image preprocessing technology: the fatigue state of the metal structure of the crane is automatically detected by an image recognition method, the expansion information of the fatigue crack of the metal structure is recorded, and the fatigue crack expansion characteristic of an important structure part can be obtained by storing and analyzing the imaged information, wherein the method comprises the steps of image preprocessing, crack characteristic extraction, recognition and the like.
Image real-time analysis technology: and automatically classifying and identifying various types of defects, and providing an intelligent defect identification algorithm which integrates multi-scale geometric analysis and a support vector machine.
With reference to fig. 2, based on the foregoing detection system, the present invention further provides a working method of a crane defect detection system based on unmanned aerial vehicle image recognition, where the working method includes the following steps:
s1, acquiring the structural morphology of the crane to be detected, and planning the flight path of the unmanned aerial vehicle; according to waiting to examine the hoist position, fix a position unmanned aerial vehicle, order about unmanned aerial vehicle and fly along planning the route.
S2, in the flying process of the unmanned aerial vehicle, an airborne detection device is adopted to collect surface structure data of the crane, the collection structure is sent to a ground console, a real-time analysis system installed in the ground console improves the resolution of the collected image by adopting image local area constraint and deconvolution image algorithms, and then a defect identification model based on a fractal threshold method is adopted to process the collected data, so that a crane surface defect evaluation result is obtained.
And S3, obtaining a visual evaluation result of the collected image by combining the adjusted image resolution and the crane surface defect evaluation result, and sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system.
S4, the unmanned aerial vehicle main controller adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console.
And S5, adjusting the fractal threshold of the defect identification model by the real-time analysis system according to the evaluation accuracy of the crane surface defects.
Further, in step S2, the process of processing the collected data by using the defect identification model based on the fractal threshold method to obtain the crane surface defect evaluation result includes the following steps:
and S21, generating a crane metal structure image library, storing a certain amount of crane surface structure image samples marked with defect types and defect positions, and denoising the crane surface structure image samples.
S22, constructing a defect identification model based on the deep learning algorithm, wherein the defect identification model comprises an image processing unit and a defect identification unit.
And S23, based on the image processing unit, performing threshold segmentation on the denoised image sample by adopting a fractal threshold method, decomposing the segmented image sample, extracting the characteristic vector of the decomposed image sample, performing dimension reduction processing on the characteristic vector, and generating a training sample.
And S24, training the defect identification unit by using the training sample.
S24, extracting the shot surface structure image of the crane to be detected, importing the image into a defect identification model, performing threshold segmentation on the denoised surface structure image by adopting a fractal threshold method, decomposing the segmented image, extracting image characteristic vectors, performing dimensionality reduction on the characteristic vectors, and then performing real-time identification and real-time positioning on the crane surface defects.
Further, the defect identification model comprises a defect accuracy rate evaluation unit, wherein the defect accuracy rate evaluation unit is used for evaluating the crane surface defect evaluation accuracy rate within a preset time range and sending the crane surface defect evaluation accuracy rate obtained through evaluation to the image processing unit so as to adjust the fractal threshold value of the image processing unit.
The system is applied to detecting the crane structure, the implementation flow of the invention is shown in figure 2, and the specific steps are as follows:
step 1: the unmanned aerial vehicle flies to the height and the position required by the operation of the large crane through the GPS device.
Step 2: data information transmitted back through the airborne detection device judges whether the visual sensor has the phenomenon of being directly irradiated by light or not, and the flight direction of the unmanned aerial vehicle is adjusted.
And step 3: the camera rotating holder drives the camera to rotate, the camera is aligned to a position to be measured of the crane to carry out multi-azimuth shooting, the pitch angle range of the vertical motion direction of the camera reaches +/-90 degrees, the azimuth motion range of the camera in the horizontal direction reaches +/-150 degrees, the camera can overlook imaging downwards and look up imaging upwards, and multi-azimuth shooting of various key positions of the large crane can be carried out without viewing field and dead angles.
And 4, step 4: and sending the image acquired by the unmanned aerial vehicle in real time to a ground command platform through a signal transceiver for analysis and processing.
And 5: as shown in fig. 3, for the problem of shake and blur of a shot image caused by attitude change and vibration of the unmanned aerial vehicle, a minimum energy function is obtained by an image deblurring shake algorithm based on image local area constraint and deconvolution, various prior constraints of an image local area and a motion blur kernel are utilized, the minimum energy function is solved by an alternative minimization method, the motion blur kernel is accurately estimated, an original clear image is restored by a deconvolution algorithm, and the image resolution of the unmanned aerial vehicle is improved.
Step 6: according to the image resolution output by preprocessing, the optimized flight direction and the shooting angle of the unmanned aerial vehicle are coarsely adjusted, and the quality of the collected image is improved.
And 7: and (3) performing threshold segmentation on the denoised image by adopting a fractal threshold method, identifying a crack region, realizing binarization of the image, establishing a crack skeleton data structure model, and performing expansion, corrosion and refinement by adopting mathematical morphology to improve the connectivity of the crack region.
And 8: decomposing the image by the contourlet transformation and the shear wave transformation respectively, extracting the statistic value of the decomposed image as a characteristic vector, and then reducing the dimension of the extracted characteristic vector by using an underwriter projection algorithm to obtain the characteristic vector after dimension reduction.
And step 9: firstly, a training process is carried out, crane metal structure images are collected through an unmanned aerial vehicle vision system, four crane metal structure image libraries of normal, crack, rust and abrasion are established, the feature vector is obtained by using the feature extraction method based on multi-scale geometric analysis and is input into a support vector machine for training.
Step 10: the method comprises the steps of acquiring images of key structure parts such as a crane girder and a boom by using an unmanned aerial vehicle vision system, carrying out primary processing on the images by using an unmanned aerial vehicle image preprocessing technology, improving the contrast and geometric precision of the images, obtaining a feature vector by using a feature extraction method based on multi-scale geometric analysis, inputting the feature vector into a support vector machine for defect classification, and carrying out intelligent detection on defects such as cracks, corrosion and abrasion.
Step 11: and adjusting a fractal threshold according to the detection accuracy, performing threshold segmentation on the denoised image again by adopting a fractal threshold method, extracting and identifying high-precision crack characteristics, and improving the detection accuracy.
Step 12: and combining the defect detection result and the image resolution to obtain a visual evaluation result of the collected image, and sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system.
Step 13: the unmanned aerial vehicle main control unit carries out the fine tuning to the direction of flight of unmanned aerial vehicle body and airborne detecting device's shooting angle according to the vision evaluation result that ground control platform returned.
In the invention, the defect identification result is also considered in fine tuning, and the defect identification result comprises the position of the surface defect relative to the crane, the defect type, the range ratio of the defect area in the image and the position of the defect area in the image. For example, if a certain defect is located at the edge of an image, the position of the unmanned aerial vehicle needs to be readjusted due to the fact that false identification is easy to occur, so that the defect is located in the central area of the image as far as possible, the defect is guaranteed to be completely displayed in the image, and the defect identification accuracy is improved. For another example, a part or a key area of the crane has multiple defects, and the occupation ratio of part of the defects in the image is too large, so that the relevance between the defects is difficult to embody, and performance evaluation on the part or the key area of the crane can be performed by combining multiple pictures, and at this time, the shooting position and the shooting parameters of the unmanned aerial vehicle can be adjusted, so that the whole image of the part or the key area can be shot by taking the defects as key points as far as possible.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (7)
1. A crane defect detection system based on unmanned aerial vehicle image recognition is characterized by comprising an unmanned aerial vehicle body, an unmanned aerial vehicle main controller, an airborne detection device, a communication link system and a ground control console;
the unmanned aerial vehicle main controller is used for controlling the flight track and the flight attitude of the unmanned aerial vehicle body;
the airborne detection device is mounted on the unmanned aerial vehicle body and used for collecting images on the surface of the crane and sending collected data to the unmanned aerial vehicle main controller; the unmanned aerial vehicle main controller also sends the acquired data to a ground console through a communication link system, a real-time analysis system installed in the ground console improves the resolution of the acquired image by adopting image local area constraint and deconvolution image algorithms, a defect identification model based on a fractal threshold method is adopted to process the acquired data to obtain a crane surface defect evaluation result, and the real-time analysis system adjusts the fractal threshold of the defect identification model according to the crane surface defect evaluation accuracy;
the real-time analysis system is also used for obtaining the visual evaluation result of the collected image by combining the adjusted image resolution and the crane surface defect evaluation result, sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system, and the unmanned aerial vehicle main controller adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console.
2. The crane defect detection system based on unmanned aerial vehicle image recognition of claim 1, wherein the defect recognition result comprises a position of the surface defect relative to the crane, a defect type, a range ratio of the defect area in the image, and a position of the defect area in the image.
3. The crane defect detection system based on unmanned aerial vehicle image recognition of claim 1, wherein the airborne detection device comprises several or all of a visible light shooting device for collecting visible light images of the crane surface structure, a laser scanning device for collecting dimension data of the crane surface structure and an infrared light shooting device for collecting temperature difference data of the crane surface.
4. The crane defect detection system based on unmanned aerial vehicle image recognition of claim 3, wherein the airborne detection device is mounted on a rotating holder, and the shooting angle is adjusted by the rotating holder, wherein the pitch angle range in the vertical motion direction is ± 90 °, and the azimuth motion range in the horizontal direction is ± 150 °.
5. The working method of the crane defect detection system based on unmanned aerial vehicle image recognition is characterized by comprising the following steps:
s1, acquiring the structural morphology of the crane to be detected, and planning the flight path of the unmanned aerial vehicle; positioning the unmanned aerial vehicle according to the position of the crane to be detected, and driving the unmanned aerial vehicle to fly along a planned path;
s2, in the flying process of the unmanned aerial vehicle, an airborne detection device is adopted to collect surface structure data of the crane, the collected structure is sent to a ground console, a real-time analysis system installed in the ground console improves the resolution of the collected image by adopting image local area constraint and deconvolution image algorithms, and then a defect identification model based on a fractal threshold method is adopted to process the collected data to obtain a crane surface defect evaluation result;
s3, obtaining a visual evaluation result of the collected image by combining the adjusted image resolution and the crane surface defect evaluation result, and sending the visual evaluation result to the unmanned aerial vehicle main controller through the communication link system;
s4, the unmanned aerial vehicle main controller adjusts the flight direction of the unmanned aerial vehicle body and the shooting angle of the airborne detection device according to the visual evaluation result returned by the ground control console;
and S5, adjusting the fractal threshold of the defect identification model by the real-time analysis system according to the evaluation accuracy of the crane surface defects.
6. The working method of the crane defect detection system based on unmanned aerial vehicle image recognition as claimed in claim 5, wherein in step S2, the step of processing the collected data by using the defect recognition model based on the fractal threshold method to obtain the crane surface defect evaluation result comprises the following steps:
s21, generating a crane metal structure image library, storing a certain amount of crane surface structure image samples marked with defect types and defect positions, and denoising the crane surface structure image samples;
s22, constructing a defect identification model based on multi-scale geometric analysis and a support vector machine, wherein the defect identification model comprises an image processing unit and a defect identification unit;
s23, based on the image processing unit, performing threshold segmentation on the denoised image sample by adopting a fractal threshold method, decomposing the segmented image sample, extracting a characteristic vector of the decomposed image sample, performing dimension reduction processing on the characteristic vector, and generating a training sample;
s24, training the defect identification unit by adopting the training sample;
s24, extracting the shot surface structure image of the crane to be detected, importing the image into a defect identification model, performing threshold segmentation on the denoised surface structure image by adopting a fractal threshold method, decomposing the segmented image, extracting image characteristic vectors, performing dimensionality reduction on the characteristic vectors, and then performing real-time identification and real-time positioning on the crane surface defects.
7. The operating method of the crane defect detection system based on unmanned aerial vehicle image recognition as claimed in claim 6, wherein the defect recognition model comprises a defect accuracy rate evaluation unit for evaluating crane surface defect evaluation accuracy rate within a preset time range and sending the evaluated crane surface defect evaluation accuracy rate to the image processing unit so as to adjust a fractal threshold of the image processing unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011133457.1A CN112348034A (en) | 2020-10-21 | 2020-10-21 | Crane defect detection system based on unmanned aerial vehicle image recognition and working method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011133457.1A CN112348034A (en) | 2020-10-21 | 2020-10-21 | Crane defect detection system based on unmanned aerial vehicle image recognition and working method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112348034A true CN112348034A (en) | 2021-02-09 |
Family
ID=74359542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011133457.1A Pending CN112348034A (en) | 2020-10-21 | 2020-10-21 | Crane defect detection system based on unmanned aerial vehicle image recognition and working method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348034A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112903026A (en) * | 2021-02-25 | 2021-06-04 | 南京翰氜信息科技有限公司 | Engineering safety remote online monitoring system based on machine vision and artificial intelligence |
| CN113428374A (en) * | 2021-07-29 | 2021-09-24 | 西南交通大学 | Bridge structure detection data collection method and unmanned aerial vehicle system |
| CN113495580A (en) * | 2021-08-06 | 2021-10-12 | 澜枢(上海)智能科技有限公司 | Unmanned aerial vehicle system for detecting crane structure of port and wharf and using method thereof |
| CN113537159A (en) * | 2021-09-09 | 2021-10-22 | 丹华海洋工程装备(南通)有限公司 | Crane risk data identification method based on artificial intelligence |
| CN113640301A (en) * | 2021-08-03 | 2021-11-12 | 江苏省特种设备安全监督检验研究院 | Bolt detection method for special equipment |
| CN113744270A (en) * | 2021-11-05 | 2021-12-03 | 南京市特种设备安全监督检验研究院 | Unmanned aerial vehicle visual detection and identification method for crane complex steel structure surface defects |
| CN114035608A (en) * | 2021-11-10 | 2022-02-11 | 广东电网能源发展有限公司 | Unmanned aerial vehicle-based inspection training method and system, training terminal and storage medium |
| CN114051095A (en) * | 2021-11-12 | 2022-02-15 | 苏州臻迪智能科技有限公司 | Remote processing method of video stream data and shooting system |
| WO2023102110A1 (en) * | 2021-12-01 | 2023-06-08 | Brown University | Computations trained machine learning models for non-destructive evaluation of material and structural flaws |
| CN116309849A (en) * | 2023-05-17 | 2023-06-23 | 新乡学院 | Crane positioning method based on visual radar |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120262708A1 (en) * | 2009-11-25 | 2012-10-18 | Cyberhawk Innovations Limited | Unmanned aerial vehicle |
| CN107380420A (en) * | 2017-08-23 | 2017-11-24 | 南京市特种设备安全监督检验研究院 | A kind of vibrative mechanism detection means and method based on unmanned plane mechanical arm |
| WO2019047338A1 (en) * | 2017-09-08 | 2019-03-14 | 上海振华重工(集团)股份有限公司 | Port machinery inspection device and inspection method |
-
2020
- 2020-10-21 CN CN202011133457.1A patent/CN112348034A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120262708A1 (en) * | 2009-11-25 | 2012-10-18 | Cyberhawk Innovations Limited | Unmanned aerial vehicle |
| CN107380420A (en) * | 2017-08-23 | 2017-11-24 | 南京市特种设备安全监督检验研究院 | A kind of vibrative mechanism detection means and method based on unmanned plane mechanical arm |
| WO2019047338A1 (en) * | 2017-09-08 | 2019-03-14 | 上海振华重工(集团)股份有限公司 | Port machinery inspection device and inspection method |
Non-Patent Citations (2)
| Title |
|---|
| 周前飞;丁树庆;冯月贵;张慎如;王会方;任诗波;: "大型起重机金属结构无人机智能视觉检测系统研究", 内燃机与配件, no. 19, pages 224 - 228 * |
| 周前飞;丁树庆;王会方;庆光蔚;胡静波;吴祥生;: "基于旋翼飞行机械臂的在役大型金属结构智能检测系统研究", 机电工程技术, no. 11 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112903026A (en) * | 2021-02-25 | 2021-06-04 | 南京翰氜信息科技有限公司 | Engineering safety remote online monitoring system based on machine vision and artificial intelligence |
| CN113428374A (en) * | 2021-07-29 | 2021-09-24 | 西南交通大学 | Bridge structure detection data collection method and unmanned aerial vehicle system |
| CN113640301B (en) * | 2021-08-03 | 2024-02-09 | 江苏省特种设备安全监督检验研究院 | Bolt detection method for special equipment |
| CN113640301A (en) * | 2021-08-03 | 2021-11-12 | 江苏省特种设备安全监督检验研究院 | Bolt detection method for special equipment |
| CN113495580A (en) * | 2021-08-06 | 2021-10-12 | 澜枢(上海)智能科技有限公司 | Unmanned aerial vehicle system for detecting crane structure of port and wharf and using method thereof |
| CN113537159A (en) * | 2021-09-09 | 2021-10-22 | 丹华海洋工程装备(南通)有限公司 | Crane risk data identification method based on artificial intelligence |
| CN113537159B (en) * | 2021-09-09 | 2021-12-03 | 丹华海洋工程装备(南通)有限公司 | Crane risk data identification method based on artificial intelligence |
| CN113744270A (en) * | 2021-11-05 | 2021-12-03 | 南京市特种设备安全监督检验研究院 | Unmanned aerial vehicle visual detection and identification method for crane complex steel structure surface defects |
| CN114035608A (en) * | 2021-11-10 | 2022-02-11 | 广东电网能源发展有限公司 | Unmanned aerial vehicle-based inspection training method and system, training terminal and storage medium |
| CN114051095A (en) * | 2021-11-12 | 2022-02-15 | 苏州臻迪智能科技有限公司 | Remote processing method of video stream data and shooting system |
| WO2023102110A1 (en) * | 2021-12-01 | 2023-06-08 | Brown University | Computations trained machine learning models for non-destructive evaluation of material and structural flaws |
| CN116309849A (en) * | 2023-05-17 | 2023-06-23 | 新乡学院 | Crane positioning method based on visual radar |
| CN116309849B (en) * | 2023-05-17 | 2023-08-25 | 新乡学院 | Crane positioning method based on visual radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112348034A (en) | Crane defect detection system based on unmanned aerial vehicle image recognition and working method | |
| Jiang et al. | Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system | |
| CN105302151B (en) | A kind of system and method for aircraft docking guiding and plane type recognition | |
| WO2018028103A1 (en) | Unmanned aerial vehicle power line inspection method based on characteristics of human vision | |
| CN105373135B (en) | A kind of method and system of aircraft docking guidance and plane type recognition based on machine vision | |
| Li et al. | Automatic bridge crack detection using Unmanned aerial vehicle and Faster R-CNN | |
| JP2019009919A (en) | Patrol inspection support system and patrol inspection support control program | |
| CN109521019A (en) | A kind of bridge bottom crack detection method based on unmanned plane vision | |
| CN111768417B (en) | Railway wagon overrun detection method based on monocular vision 3D reconstruction technology | |
| CN107796373B (en) | Distance measurement method based on monocular vision of front vehicle driven by lane plane geometric model | |
| Máthé et al. | Vision-based control of a quadrotor for an object inspection scenario | |
| CN111598846B (en) | Method for detecting rail defects in tunnel based on YOLO | |
| CN108535789A (en) | A kind of foreign matter identifying system based on airfield runway | |
| AU2019222803A1 (en) | Volume measurement apparatus and method | |
| Savva et al. | ICARUS: Automatic autonomous power infrastructure inspection with UAVs | |
| CN108564628A (en) | A kind of cutterhead vision positioning orientation system towards development machine automation | |
| CN114812398B (en) | High-precision real-time crack detection platform based on unmanned aerial vehicle | |
| CN114004814A (en) | Coal gangue identification method and system based on deep learning and gray scale third moment analysis | |
| Liu et al. | Framework for automated UAV-based inspection of external building façades | |
| CN107767366B (en) | A kind of transmission line of electricity approximating method and device | |
| CN117274845A (en) | Method, system, equipment and storage medium for capturing images of flying unmanned aerial vehicle | |
| CN117115249A (en) | Container lock hole automatic identification and positioning system and method | |
| CN116243725A (en) | Substation unmanned aerial vehicle inspection method and system based on visual navigation | |
| Budzan | Fusion of visual and range images for object extraction | |
| CN112560922A (en) | Vision-based foggy-day airplane autonomous landing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |