CN112966576B - System and method for aiming insulator water washing robot based on multi-light source image - Google Patents
System and method for aiming insulator water washing robot based on multi-light source image Download PDFInfo
- Publication number
- CN112966576B CN112966576B CN202110203773.XA CN202110203773A CN112966576B CN 112966576 B CN112966576 B CN 112966576B CN 202110203773 A CN202110203773 A CN 202110203773A CN 112966576 B CN112966576 B CN 112966576B
- Authority
- CN
- China
- Prior art keywords
- insulator
- image
- water washing
- washing robot
- aiming
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 239000012212 insulator Substances 0.000 title claims abstract description 95
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 79
- 238000005406 washing Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000001514 detection method Methods 0.000 claims abstract description 26
- 230000000007 visual effect Effects 0.000 claims abstract 2
- 238000012549 training Methods 0.000 claims description 21
- 239000011159 matrix material Substances 0.000 claims description 16
- 230000008859 change Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000013136 deep learning model Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000009434 installation Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 abstract 1
- 238000004140 cleaning Methods 0.000 description 5
- 239000003344 environmental pollutant Substances 0.000 description 5
- 231100000719 pollutant Toxicity 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 238000011010 flushing procedure Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000003373 anti-fouling effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000010292 electrical insulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009422 external insulation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009413 insulation Methods 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000005507 spraying Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B08—CLEANING
- B08B—CLEANING IN GENERAL; PREVENTION OF FOULING IN GENERAL
- B08B3/00—Cleaning by methods involving the use or presence of liquid or steam
- B08B3/02—Cleaning by the force of jets or sprays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Insulators (AREA)
Abstract
The invention discloses an insulator water washing robot aiming system and method based on multiple light source images, and the system comprises a binocular camera, an infrared camera, an industrial control computer and a water washing robot; the method comprises the steps that a binocular camera collects visible light images of insulators, three-dimensional coordinates of the insulators are located according to the parallax principle, an infrared camera collects infrared images of the insulators, dirt degree detection is conducted on the insulators, the aiming direction of a washing robot is determined according to the three-dimensional coordinates and the dirt degree state of the insulators, an industrial control computer feeds back position information of the insulators needing to be aimed to a washing robot controller, joint movement of the washing robot is controlled, and visual servo control of the washing robot is achieved. The invention enables the characteristic advantages of different image sources to be complementary by fusing the visible light image and the infrared image, can effectively avoid the one-sidedness of single information, and enables the aiming result of the water washing robot to be more accurate and effective.
Description
Technical Field
The invention belongs to the field of automatic control of water washing, and particularly relates to an insulator water washing robot aiming system and method based on multiple light source images.
Background
The topological structure of the power grid in China is complex, the environment pollution source has diversity and variability, and the guarantee of the reliable operation of the power grid equipment is an important ring for power overhaul, operation and maintenance. The insulator is used as a key part for playing a role in mechanical support and electrical insulation in a railway contact network, is one of important components of the whole power grid system, is used in an outdoor environment for a long time, the surface of a material is very easily polluted by various pollutants in the air, pollutant particles are adsorbed on the surface of the insulator material under the action of cross action of external force of the environment, a large number of pollutants are accumulated in the long term to finally form a pollutant layer, and the pollutants are different in components due to different regional environments and usually mainly comprise a large number of dust, metal salt particles and the like. If in the stage of dirt accumulation on the surface of the external insulation, the dirt state of the equipment is monitored in time, and the equipment is cleaned in a targeted manner, so that the probability of flashover can be effectively reduced, and the economic loss caused by the flashover can be reduced.
Charged water flushing is commonly used as one of measures for electrical equipment to effectively prevent a pollution flashover accident. And under the condition of uninterrupted power supply, the water column with high-pressure insulation reaching a certain value is used for washing the dirt deposited on the surface of the insulator. The high pressure generated by the high pressure pump system is utilized to form a cleaning medium with strong impact force in a spraying mode, and the cleaning medium impacts the surface of the insulator of the contact net to force dirt on the surface of the insulator to fall off. The charged water washing has the advantages of no interruption of power supply, no equipment power failure, and no reversing and closing operations of the equipment; the cleaning device is not limited by the power-off time of the power equipment, can clean dirty equipment with water at any time, and has good cleaning effect. At present, insulator live-line water washing operation mostly adopts manual mode, and operating personnel need work near high-voltage electric equipment, and handheld washing device washes the insulator, and the operational environment is dangerous relatively, and work load is very big, has seriously influenced the efficiency of water washing operation. In recent years, the development of automatic cleaning and anti-fouling technology and equipment for insulators at home and abroad becomes a research hotspot.
At present, the problem that insulator water washing robot exists is:
1. a single vision positioning method is adopted. The visible light image has higher contrast and resolution, but is easily influenced by the illumination environment; the infrared image can measure the surface temperature of the insulator and generate an infrared thermography, but can be influenced by factors such as atmospheric thermal radiation and the like. The existing insulator water washing robot only depends on one of a visible light image or an infrared image to position the insulator, so that the accuracy of insulator identification is influenced, and the washing efficiency is reduced.
2. Although a certain degree of automation device is introduced, an operator is still required to control the water washing operation of the robot, the intelligent level is low, and the intelligent requirement of the water washing operation cannot be met.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides an insulator water washing robot aiming system and method based on multiple light source images.
The invention relates to an insulator water washing robot aiming system based on multiple light source images, which comprises a binocular camera, an infrared camera, an industrial control computer and a water washing robot; the water washing robot comprises a water gun and a two-degree-of-freedom mechanical arm, wherein the mechanical arm can drive the water gun to horizontally and vertically rotate, and the water gun is connected with water production equipment through an insulated water pipe; binocular camera and infrared camera parallel mount are in the rear portion top of the squirt of the robot is washed to water, the installation direction is unanimous with the squirt direction, on-the-spot visible light image and the infrared image are gathered respectively to binocular camera and infrared camera, image information passes through wireless router, wireless network card conveys the industrial control computer, the industrial control computer is used for calculating the squirt direction of aiming, and with information feedback to the robot control ware is washed to water, the robot motion is washed to control water, thereby realize the insulator and wash the vision servo control of robot.
The invention discloses an aiming method of an insulator water washing robot based on multiple light source images, which is characterized in that the aiming system of the insulator water washing robot based on the multiple light source images comprises the following steps:
step 1: and respectively collecting visible light images and infrared images of the insulator, and enlarging the training sample by a data enhancement method.
And 2, step: by using AFK-MC 2 And performing clustering analysis on the insulator marking frame by using a clustering algorithm to determine the size of the insulator prior frame.
And step 3: and inputting the image sample into a deep learning model based on YOLOv4 for training, and establishing the deep learning model for insulator identification and positioning and pollution degree detection.
And 4, step 4: establishing a jacobian matrix J of the mechanical arm q And the Jacobian matrix J of the image r Determining the change rate of the insulator image feature space
Angular velocity of two rotary joints of water washing robot
The relationship between them, which is expressed as follows:
and 5: and (3) acquiring a field real-time image through a binocular camera and an infrared camera, splicing the visible light image and the infrared image acquired at the same time into one image, and inputting the image into the model obtained in the step (3) to perform insulator identification and positioning and dirt degree judgment.
Step 6: and reducing the three-dimensional coordinates of the contaminated insulator by using the binocular camera parallax principle, and calculating the distance between the insulator and the binocular camera according to the coordinates.
And 7: the distance obtained in the step 6 is brought into a Jacobian matrix, two corner angular velocities of the water washing robot are calculated according to the change rate of the insulator in the image characteristic space and the Jacobian matrix, the two corner angular velocities are sent to a robot controller, the joint of the water washing robot is controlled to move, and the aiming of the insulator is completed; the calculation formula of the change rate of the insulator in the image characteristic space is as follows:
wherein (DstX, DstY) is the coordinate of the center point of the image of the binocular camera, (SrcX, SrcY) is the coordinate of the center point of the insulator, and sigma is a proportionality coefficient.
Further, in step 1, the image enhancement method includes flipping, rotating, clipping, translating, adding gaussian noise and contrast transformation to expand the training sample.
Further, the step 2 comprises the following specific steps: randomly selecting an object as an initial cluster center, respectively calculating the probability of the next selection of the object for each of the rest objects according to the IOU distance from the initial cluster center, then constructing 8 Markov chains with a certain length according to the probability, wherein the last point of the Markov chain is a new cluster center, adding the initial point, and finally obtaining 9 cluster centers which are respectively used as 9 standard box sizes and finally determined as [101,149], [122,51], [323,254], [36,76], [75,53], [72,146], [12,16], [18,36], [41,30 ].
Further, yoolov 4 in step 3 is a target detection network with a backbone network of CSPDarkNet53, and there are 161 network layers in total, of which 3 are output feature layers for multi-scale feature detection, and the sizes are 19 × 19, 38 × 38, and 76 × 76, respectively.
Further, the model training in the step 3 comprises the following steps:
s31, pre-marking insulators in the visible light image sample set, and pre-marking normal insulators and insulators with different pollution degrees in the infrared image sample set;
s32, inputting the sizes of the 9 clustered prior frames into a backbone network CSPDarkNet53 of YOLOv 4;
s33, performing feature processing of four different scales on the input sample through the SPPNet network;
s34, performing information fusion of the shallow feature and the deep feature on the sample through a PANet path aggregation network to obtain fusion features of different scales;
s35, outputting training results through three feature layers;
s36, calculating a loss function according to the output result, dynamically adjusting the learning rate and the batch processing size, and stopping training until the loss function value output by the training set is smaller than a preset threshold value or reaches a set maximum iteration number to obtain a trained network model, namely a prediction model.
Further, the insulator positioning and pollution degree judgment in the step 5 comprises the following steps:
s51, splicing the two visible light images collected by the binocular camera and the infrared image collected by the infrared camera into one image, and inputting the image into a trained YOLOv4 detection model.
S52, feature detection and extraction are carried out on the 19 x 19 feature map by adopting three large-size prior frames, feature detection and extraction are carried out on the 38 x 38 feature map by adopting three moderate-size prior frames, and feature detection and extraction are carried out on the 76 x 76 feature map by adopting three small-size prior frames.
S53, decoding the output results of the three characteristic layers, outputting positioning and filthy degree information, and completing identification and positioning.
The beneficial technical effects of the invention are as follows:
1. according to the method, a large number of insulator visible light image samples and infrared image samples are acquired, and an insulator pollution degree identification database under different background interferences is constructed aiming at various interference problems of complex background interferences, various shooting angles, various illumination conditions and the like in the insulator detection process. Visible light images and infrared images are respectively collected by the binocular camera and the infrared camera, the insulators are identified through the YOLOv4 deep learning detection model, and coordinate information and pollution states of the insulators are determined, so that identification and positioning of the pollution states of the insulators based on multi-light source image information are achieved, the aiming position of the water gun can be determined, and the accuracy of flushing operation is effectively improved.
2. The binocular camera and the infrared camera are connected with the industrial control computer through the wireless transmitting and receiving device, image data are transmitted to the industrial control computer in real time, the computer identifies the positioning insulator and the insulator pollution degree in the image through the YOLOv4 deep learning detection model, the three-dimensional coordinates of the insulator are accurately reduced through the parallax principle, the motion of the water washing robot is controlled, the degree of autonomy of the robot is effectively improved, and the great effect is achieved on the improvement of the washing efficiency of the robot.
Drawings
Fig. 1 is a schematic view of the aiming system of the insulator water washing robot based on multiple light source images.
Fig. 2 is a flowchart of the aiming method of the insulator water washing robot based on multi-light source images.
Fig. 3 is a schematic diagram of a water washing robot vision servo control system.
Fig. 4 is a schematic diagram of a network structure of YOLOv 4.
Detailed Description
The invention is described in further detail below with reference to the figures and the detailed description.
The insulator water washing robot aiming system based on the multi-light source image is shown in figure 1 and comprises a binocular camera, an infrared camera, an industrial control computer and a water washing robot; the water washing robot comprises a water gun and a two-degree-of-freedom mechanical arm, wherein the mechanical arm can drive the water gun to horizontally and vertically rotate, and the water gun is connected with water production equipment through an insulated water pipe; binocular camera and infrared camera parallel mount are in the rear portion top of the squirt of the robot is washed to water, the installation direction is unanimous with the squirt direction, on-the-spot visible light image and the infrared image are gathered respectively to binocular camera and infrared camera, image information passes through wireless router, wireless network card conveys the industrial control computer, the industrial control computer is used for calculating the squirt direction of aiming, and with information feedback to the robot control ware is washed to water, control water washes the robot motion, thereby realize that the insulator water washes the vision servo control of robot (as shown in figure 3).
The method for aiming the insulator water washing robot based on the multi-light source image is shown in figure 2 and specifically comprises the following steps:
1. the method comprises the steps of collecting visible light images and infrared images of the insulator, and expanding a training sample through a data enhancement method, wherein the image enhancement method comprises the steps of turning, rotating, cutting, translating, adding Gaussian noise, converting contrast and the like. The generalization capability of the model can be enhanced by expanding the training samples, and the accuracy of the model in identifying the target in a complex background and interference is improved.
2. Setting a prior frame of a YOLOv4 deep learning model according to characteristics of insulator samples, and adopting AFK-MC 2 And performing clustering analysis on the insulator marking frame in the image sample by using a clustering algorithm to determine the size of the prior frame. The conventional k-Means clustering algorithm, like other non-convex optimization algorithms, may converge to a local minimum and thus may not obtain an optimal value. And AFK-MC 2 The clustering algorithm can obtain better clustering results without assuming data distribution. For large data sets, from 0 toAFK-MC at a relative error of 1% 2 The algorithm is 200 to 1000 times faster than the k-Means algorithm. The method comprises the following specific steps: randomly selecting an object as an initial clustering center, respectively calculating the probability of the next time of selection of the object for each of the rest objects according to the IOU distance from the initial clustering center, and then constructing 8 Markov chains with certain length according to the probability, wherein the last point of the Markov chain is a new clustering center. Adding an initial point, finally obtaining 9 cluster centers, namely the obtained 9 prior box sizes, and finally determining as [101,149]]，[122,51]，[323,254]，[36,76]，[75,53]，[72,146]，[12,16]，[18,36]，[41,30]。
3. And (3) inputting the image sample into a deep learning model based on YOLOv4 for training, and establishing the deep learning model for insulator recognition and fault detection. YOLOv4 is a target detection network with a backbone network of CSPDarkNet53, and has 161 network layers, of which 3 are output feature layers for multi-scale feature detection, and the sizes are 19 × 19, 38 × 38 and 76 × 76, respectively. YOLOV4 is an improved version of YOLOV3, improves recognition accuracy under the condition of ensuring speed, can basically ensure real-time operation on an industrial control computer, and has a network structure as shown in fig. 4.
The model training comprises the following steps:
s31, pre-labeling insulators in the visible image sample set, and pre-labeling normal insulators and insulators with different pollution degrees in the infrared image sample set;
s32, inputting the sizes of the 9 prior frames obtained by clustering into a CSPDarkNet53 of a backbone network of YOLOv 4;
s33, performing feature processing of four different scales on the input sample through the SPPNet network;
s34, performing information fusion of the shallow feature and the deep feature on the sample through a PANet path aggregation network to obtain fusion features of different scales;
s35, outputting training results through three feature layers;
s36, calculating a loss function according to the output result, dynamically adjusting the learning rate and the batch processing size, and stopping training until the loss function value output by the training set is smaller than a preset threshold value or reaches a set maximum iteration number to obtain a trained network model, namely a prediction model.
4. Establishing jacobi matrix J of mechanical arm q And image Jacobian matrix J r Determining the change rate of the insulator image feature space
At two angular velocities of rotation with the flushing device
The relationship between them, which is expressed as follows:
J q is a jacobian matrix of the mechanical arm and represents the relationship between the generalized velocity of the tail end of the washing device in space and the angular velocity of two corners of the washing device, J r The relationship between the change rate of the image feature space and the tail end speed of the mechanical arm is described as an image Jacobian matrix.
Mechanical jacobian matrix J q The calculation formula is as follows:
wherein theta is 1 、θ 2 Two-degree-of-freedom mechanical arm joint angle a of water washing robot 1 The arm link length is parallel to the camera optical axis.
Image jacobian matrix J r The calculation formula is as follows:
wherein f is the focal length of the binocular camera, and (X, Y, Z) is the coordinate of the target in the camera coordinate system.
5. Acquiring a field real-time image through a binocular camera and an infrared camera, splicing a visible light image and an infrared image acquired at the same time into one image, and inputting the image into the model obtained in the step 3 to perform insulator positioning and fault diagnosis;
the insulator positioning and pollution degree judging method comprises the following steps:
s51, splicing two visible light images acquired by a binocular camera and an infrared image acquired by an infrared camera into one image, and inputting the image into a trained YOLOv4 detection model;
s52, performing feature detection and extraction on the 19 x 19 feature map by adopting three large-size prior frames, performing feature detection and extraction on the 38 x 38 feature map by adopting three moderate-size prior frames, and performing feature detection and extraction on the 76 x 76 feature map by adopting three small-size prior frames;
s53, decoding the output results of the three characteristic layers, outputting positioning and filthy degree information, and completing identification and positioning.
6. And calculating the three-dimensional coordinate of the fault insulator relative to the camera by using the binocular camera parallax principle, and calculating the distance between the insulator and the binocular camera according to the coordinate.
7. The distance obtained in the last step is substituted into a Jacobian matrix, and an industrial control computer carries out calculation according to the change rate of the insulator in the image characteristic space
Calculating two angular velocities of the water washing robot by the Jacobian matrix
And sending the signal to a robot controller, controlling the joint of the water washing robot to move, and finishing the insulator tracking.
Claims (6)
1. The method is characterized in that the used insulator water washing robot aiming system based on the multi-light source images comprises a binocular camera, an infrared camera, an industrial control computer and a water washing robot; the water washing robot comprises a water gun and a two-degree-of-freedom mechanical arm, wherein the mechanical arm can drive the water gun to horizontally and vertically rotate, and the water gun is connected with water production equipment through an insulated water pipe; the system comprises a water washing robot, a binocular camera and an infrared camera, wherein the binocular camera and the infrared camera are installed above the rear part of a water gun of the water washing robot in parallel, the installation direction is consistent with the direction of the water gun, the binocular camera and the infrared camera respectively collect on-site visible light images and infrared images, image information is transmitted to an industrial control computer through a wireless router and a wireless network card, the industrial control computer is used for calculating the aiming direction of the water gun and feeding the information back to a water washing robot controller to control the water washing robot to move, and therefore visual servo control of the insulator water washing robot is achieved; the method specifically comprises the following steps:
step 1: respectively collecting visible light images and infrared images of the insulator, and enlarging a training sample by a data enhancement method;
step 2: by using AFK-MC 2 Performing clustering analysis on the insulator marking frame by using a clustering algorithm, and determining the size of the insulator prior frame;
and step 3: inputting the image sample into a deep learning model based on YOLOv4 for training, and establishing a deep learning model for insulator identification and positioning and pollution degree detection;
and 4, step 4: establishing a jacobian matrix J of the mechanical arm q And the Jacobian matrix J of the image r Determining the change rate of the insulator image feature space
Angular velocity of two rotating joints of water washing robot
The relationship between them, which is expressed as follows:
and 5: acquiring a field real-time image through a binocular camera and an infrared camera, splicing a visible light image and an infrared image acquired at the same time into one image, and inputting the image into the model obtained in the step (3) to perform insulator identification and positioning and dirt degree judgment;
and 6: restoring three-dimensional coordinates of the polluted insulator by using a binocular camera parallax principle, and calculating the distance between the insulator and a binocular camera according to the coordinates;
and 7: substituting the distance obtained in the step 6 into a Jacobian matrix, calculating two corner angular velocities of the water washing robot according to the change rate of the insulator in the image characteristic space and the Jacobian matrix, sending the two corner angular velocities to a robot controller, controlling the joint of the water washing robot to move, and finishing the aiming of the insulator; the calculation formula of the change rate of the insulator in the image characteristic space is as follows:
wherein (DstX, DstY) is the coordinate of the center point of the image of the binocular camera, (SrcX, SrcY) is the coordinate of the center point of the insulator, and sigma is a proportionality coefficient.
2. The method for aiming the insulator water washing robot based on the multi-light source image is characterized in that in the step 1, the image enhancement method comprises overturning, rotating, cutting, translating, adding Gaussian noise and contrast transformation so as to expand the training sample.
3. The aiming method of the insulator water washing robot based on the multi-light source image is characterized in that the step 2 specifically comprises the following steps: randomly selecting an object as an initial cluster center, respectively calculating the probability of the next selection of the object for each of the rest objects according to the IOU distance from the initial cluster center, then constructing 8 Markov chains with a certain length according to the probability, wherein the last point of the Markov chain is a new cluster center, adding the initial point, and finally obtaining 9 cluster centers which are respectively used as 9 standard box sizes and finally determined as [101,149], [122,51], [323,254], [36,76], [75,53], [72,146], [12,16], [18,36], [41,30 ].
4. The aiming method for the insulator water washing robot based on the multi-light source image is characterized in that the YOLOv4 in the step 3 is a target detection network with a main network of CSPDarkNet53, and the target detection network has 161 network layers in total, wherein 3 of the network layers are output feature layers used for multi-scale feature detection, and the sizes of the output feature layers are 19 × 19, 38 × 38 and 76 × 76 respectively.
5. The aiming method of the insulator water washing robot based on the multi-light source image is characterized in that the model training in the step 3 comprises the following steps:
s31, pre-marking insulators in the visible light image sample set, and pre-marking normal insulators and insulators with different pollution degrees in the infrared image sample set;
s32, inputting the sizes of the 9 prior frames obtained by clustering into a CSPDarkNet53 of a backbone network of YOLOv 4;
s33, performing feature processing of four different scales on the input sample through the SPPNet network;
s34, performing information fusion of the shallow feature and the deep feature on the sample through a PANet path aggregation network to obtain fusion features of different scales;
s35, outputting training results through three feature layers;
s36, calculating a loss function according to the output result, dynamically adjusting the learning rate and the batch processing size, and stopping training until the loss function value output by the training set is smaller than a preset threshold value or reaches a set maximum iteration number to obtain a trained network model, namely a prediction model.
6. The aiming method of the insulator water washing robot based on the multi-light source image is characterized in that the insulator positioning and filthy degree judgment in the step 5 comprises the following steps:
s51, splicing two visible light images acquired by a binocular camera and an infrared image acquired by an infrared camera into one image, and inputting the image into a trained YOLOv4 detection model;
s52, carrying out feature detection and extraction on the 19 x 19 feature map by adopting three prior frames with larger sizes, carrying out feature detection and extraction on the 38 x 38 feature map by adopting three prior frames with moderate sizes, and carrying out feature detection and extraction on the 76 x 76 feature map by adopting three prior frames with smaller sizes;
s53, decoding the output results of the three characteristic layers, outputting positioning and filthy degree information, and completing identification and positioning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110203773.XA CN112966576B (en) | 2021-02-24 | 2021-02-24 | System and method for aiming insulator water washing robot based on multi-light source image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110203773.XA CN112966576B (en) | 2021-02-24 | 2021-02-24 | System and method for aiming insulator water washing robot based on multi-light source image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112966576A CN112966576A (en) | 2021-06-15 |
| CN112966576B true CN112966576B (en) | 2022-09-16 |
Family
ID=76285799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110203773.XA Expired - Fee Related CN112966576B (en) | 2021-02-24 | 2021-02-24 | System and method for aiming insulator water washing robot based on multi-light source image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112966576B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114392964B (en) * | 2021-12-24 | 2023-03-14 | 西南交通大学 | Insulator rinse-system that dirty degree of intellectuality was judged |
| CN114638883B (en) * | 2022-03-09 | 2023-07-14 | 西南交通大学 | Visual limited repositioning target method for insulator water flushing robot |
| CN114834325A (en) * | 2022-03-28 | 2022-08-02 | 西南交通大学 | Subway third rail insulator washs and detection device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203031618U (en) * | 2013-01-29 | 2013-07-03 | 山东电力集团公司电力科学研究院 | Vision system used for high-voltage live line working robot |
| CN105261029A (en) * | 2015-11-20 | 2016-01-20 | 中国安全生产科学研究院 | Method and robot for performing fire source location and fire extinguishment based on binocular vision |
| CN111223133A (en) * | 2020-01-07 | 2020-06-02 | 上海交通大学 | Registration method of heterogeneous images |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104834293B (en) * | 2015-04-28 | 2017-09-22 | 山东鲁能智能技术有限公司 | The method of transformer station's water flushing device people's vision collimation system |
| CN106596579A (en) * | 2016-11-15 | 2017-04-26 | 同济大学 | Insulator contamination condition detection method based on multispectral image information fusion |
| CN106680285B (en) * | 2016-11-17 | 2022-04-05 | 同济大学 | Method for recognizing insulator contamination state based on infrared image assisted visible light image |
| CN107507172A (en) * | 2017-08-08 | 2017-12-22 | 国网上海市电力公司 | Merge the extra high voltage line insulator chain deep learning recognition methods of infrared visible ray |
| CN112001260A (en) * | 2020-07-28 | 2020-11-27 | 国网湖南省电力有限公司 | Cable trench fault detection method based on infrared and visible light image fusion |
-
2021
- 2021-02-24 CN CN202110203773.XA patent/CN112966576B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203031618U (en) * | 2013-01-29 | 2013-07-03 | 山东电力集团公司电力科学研究院 | Vision system used for high-voltage live line working robot |
| CN105261029A (en) * | 2015-11-20 | 2016-01-20 | 中国安全生产科学研究院 | Method and robot for performing fire source location and fire extinguishment based on binocular vision |
| CN111223133A (en) * | 2020-01-07 | 2020-06-02 | 上海交通大学 | Registration method of heterogeneous images |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112966576A (en) | 2021-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112966576B (en) | System and method for aiming insulator water washing robot based on multi-light source image | |
| CN111530203B (en) | Intelligent dust measurement and control system and dust suppression and dust fall method thereof | |
| CN101625573B (en) | Digital signal processor based inspection robot monocular vision navigation system | |
| CN110908370A (en) | Unmanned inspection task planning method and system for thermal power plant | |
| CN107765145B (en) | Automatic partial discharge detection device, system and method | |
| CN106227215A (en) | Robot automatic photovoltaic module purging system and cleaning method thereof | |
| CN108363394A (en) | A kind of security robot intelligent inspection system and its method for inspecting | |
| CN103389738A (en) | Method and device for table tennis robot to predict table tennis ball trajectory | |
| CN106129889A (en) | A kind of insulator contamination on-Line Monitor Device and insulator decontamination apparatus | |
| CN107315410A (en) | A kind of automatic troubleshooting method of robot | |
| CN114179056B (en) | GIS indoor multi-degree-of-freedom rail-mounted inspection robot and application method thereof | |
| CN115641553B (en) | Online detection device and method for invaders in heading machine working environment | |
| WO2022001760A1 (en) | Remote monitorable multi-axis cooperative intelligent controller based on 5g technology | |
| CN112605033A (en) | Photovoltaic panel posture recognition and cleaning regulation and control device and method | |
| CN112282787B (en) | Multi-arm robot for automatic maintenance of tunnel and control method thereof | |
| CN107962589A (en) | A kind of substation equipment living water washing robot insulating protection system based on Multi-sensor Fusion | |
| CN112329584A (en) | Method, system and equipment for automatically identifying foreign matters in power grid based on machine vision | |
| CN116231504A (en) | Remote intelligent inspection method, device and system for booster station | |
| CN110899183A (en) | Transformer substation insulator live cleaning robot system and method | |
| CN110640744A (en) | Industrial robot with fuzzy control of motor | |
| CN112530267B (en) | Intelligent mechanical arm teaching method based on computer vision and application | |
| CN116388669A (en) | Photovoltaic panel foreign matter detection and cleaning method based on Swin transducer | |
| Zhu et al. | Road crack acquisition and analysis system based on mobile robot and deep learning | |
| CN112198472A (en) | Online remote calibration method and system for partial discharge intelligent sensor | |
| CN116842830A (en) | Motor train unit surface cleaning method based on machine learning and data driving digital twin platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220916 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |