CN114240982A - High-precision operation method of electrified maintenance robot for high-altitude settlement environment - Google Patents
High-precision operation method of electrified maintenance robot for high-altitude settlement environment Download PDFInfo
- Publication number
- CN114240982A CN114240982A CN202111452665.2A CN202111452665A CN114240982A CN 114240982 A CN114240982 A CN 114240982A CN 202111452665 A CN202111452665 A CN 202111452665A CN 114240982 A CN114240982 A CN 114240982A
- Authority
- CN
- China
- Prior art keywords
- bolt
- image
- hardware
- camera
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012423 maintenance Methods 0.000 title claims abstract description 11
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 239000013598 vector Substances 0.000 claims description 32
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 239000013589 supplement Substances 0.000 claims description 6
- 238000011524 similarity measure Methods 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 239000000047 product Substances 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 4
- 230000036544 posture Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 2
- 239000010931 gold Substances 0.000 description 2
- 229910052737 gold Inorganic materials 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000012212 insulator Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02G—INSTALLATION OF ELECTRIC CABLES OR LINES, OR OF COMBINED OPTICAL AND ELECTRIC CABLES OR LINES
- H02G1/00—Methods or apparatus specially adapted for installing, maintaining, repairing or dismantling electric cables or lines
- H02G1/02—Methods or apparatus specially adapted for installing, maintaining, repairing or dismantling electric cables or lines for overhead lines or cables
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Optimization (AREA)
- Evolutionary Biology (AREA)
- Algebra (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
- Manipulator (AREA)
Abstract
The application relates to a high-precision operation method of an electrified maintenance robot for an aerial settlement environment, wherein the robot comprises a mechanical arm, an RGB-D camera, a bolt replacing tool, a direct-current power supply and a bolt placing tool box, and the operation method comprises the following specific steps: the robot moves to a working point; acquiring an RGB-D image of the hardware; recognizing bolts and calculating postures; the bolt tool is opposite to the hardware fitting; error adjustment based on manual monitoring; and (5) carrying out hardware maintenance operation. The invention solves the problem that the final operation fails because the platform is settled due to the forward extension of the operation mechanical arm after the robot system in the air finishes the target positioning, thereby introducing settlement errors into the visual positioning data.
Description
Technical Field
The application relates to the field of electrified overhaul robots, in particular to a high-precision operation method of an electrified overhaul robot for an aerial settlement environment.
Background
With the rapid development of economy in China, people have greater and greater requirements on electric energy, and the scale of a power grid is rapidly increased. The transformer substation is used as an important ring of a power grid, and related electrical equipment (such as bus fittings and insulators) is located outdoors for a long time, so that the conditions of corrosion, heating, dirt accumulation and the like are easy to occur. In order to reduce the power failure times of the power grid and ensure normal production of the society and normal life of people, the equipment needs to be overhauled without power failure. However, most of the existing transformer substation equipment is mainly overhauled manually, and meanwhile, the operation needs to be carried out in a power failure environment, so that a great amount of economic loss is caused. Meanwhile, manual maintenance needs to be carried out in an aerial working environment, physical consumption is large, the operation difficulty is high, and meanwhile, the falling risk exists. Under the condition, live working is carried out by using the robot technology, and the method has important significance for non-power-off maintenance and personal safety of related equipment of the transformer substation.
In the technology of the robot for overhauling the hardware bolt in an electrified way, the precision requirement of the tooling motor on the hardware bolt is higher (within +/-2 mm). However, when the robot at the tail end of the bucket arm vehicle finishes the positioning of the target bolt and moves to the front of the target, the gravity center of the robot platform moves forwards to cause the settlement, so that a settlement error (4-8mm) is introduced into visual data, and finally the tooling motor is not mounted on the fitting bolt or vertical stress is generated after the tooling motor is mounted on the fitting bolt, so that the bolt is clamped in the fitting bolt hole during the unloading process, and finally the fitting overhauling operation is failed.
Disclosure of Invention
An object of the embodiment of the application is to provide a high-precision operation method for an aerial settlement environment-oriented live-line maintenance robot, and the problem that after a robot system in the aerial is used for positioning a target, an operation mechanical arm stretches forwards to cause settlement of a platform, so that a settlement error is introduced for visual positioning data, and final operation fails is solved.
In order to achieve the above purpose, the present application provides the following technical solutions:
in a first aspect, an embodiment of the present application provides a high-precision operation method for an aerial settlement environment-oriented live-line maintenance robot, where the robot includes a mechanical arm, an RGB-D camera, a bolt replacement tool, a dc power supply, and a bolt placement tool box, and the operation method includes the following specific steps:
the robot moves to a working point, and the insulating bucket arm vehicle sends the robot to the position right in front of the target hardware fitting;
acquiring an RGB-D image of the hardware, starting an RGB-D camera, directly acquiring a color image of a target hardware by the RGB camera, creating an infrared light spot of an area array on the surface of the target hardware by a structured light generator, acquiring left and right images by a binocular camera, performing rapid stereo matching on feature points of the left and right images, calculating a depth image of the target hardware according to a triangulation distance measuring principle, and finally acquiring the RGB-D image of the target hardware;
the method comprises the steps of bolt identification and posture calculation, wherein based on the hexagon characteristics of bolts, bolt identification is carried out by adopting a shape template matching algorithm, the spatial posture of the bolts is calculated by adopting a least square method by extracting a planar 3D point cloud in the middle of four bolts, and the posture of a hardware fitting plane under a camera coordinate system is obtained;
the bolt tool is opposite to the hardware fitting, the movement of the mechanical arm is decomposed, and the bolt tool at the tail end of the mechanical arm is successfully aligned to the plane of the hardware fitting;
based on error adjustment of manual monitoring, a circle mark is marked on a picture, right opposite to the bolt, of the tool on a monitoring picture, the bolt head is fitted, and position control of the mechanical arm is performed according to the circle mark, so that settlement errors are compensated, and alignment of a central shaft of the bolt tool and the bolt head is realized;
and (4) carrying out maintenance operation on the hardware fitting after the central shaft of the bolt tool is aligned with the bolt head by the robot.
In the hardware fitting RGB-D image obtaining step, the RGB-D camera further comprises a light supplement lamp, and the light supplement lamp is used for enabling the RGB camera to directly obtain the color image of the target hardware fitting under the dark light condition.
3. The high-precision operation method of the electrified overhaul robot facing the high-altitude settlement environment as claimed in claim 1, wherein in the bolt identification and posture calculation step, the concrete steps of adopting a shape template matching algorithm to identify the bolt are as follows:
hardware image I in RGB-D image of target hardware0From the hardware image I0In acquiring bolt image I1；
Bolt image I1Performing edge feature extraction processing to obtain processed image M1From a series of points pi＝(ri,ci)TAnd a corresponding direction vector di＝(ui,vi)TN, wherein:
i represents a bolt feature image M1I 1,2,.., n;
②riand ciRespectively representing bolt feature images M1The ith characteristic point is located in a row and a column;
③uiand viRespectively representing bolt feature images M1The ith characteristic point corresponds to the horizontal component and the vertical component of the direction vector;
t represents the matrix transposition;
transforming M using a second order standard rotation matrix A1And separating the translated portion to obtain a rotated component p'i(p′i＝Api) And a corresponding direction vector d'i(d′i＝Adi) From the hardware image I0Acquiring a processed image M0；
Suppose that at M0One point q ═ (x, y)TAnd the corresponding direction vector is e(x,y)＝(U(x,y),V(x,y))TWherein U is(x,y)Representing the horizontal component of the direction vector, V(x,y)The vertical component is represented. When bolt template image I1On hardware original image I0Q ═ a certain position (x, y)TWhen a match is made, the process can be converted to computing M1At M0The sum of the dot products of the normalized vectors at the previous point q (x, y), i.e. the similarity measure s, is calculated as a formulaicShown below:
in the above formula:
s represents a bolt feature image M1Hardware fitting characteristic image M0Last point q ═ x, y)TSimilarity metric values in matching; x represents the abscissa of the matching position q point, and y represents the ordinate thereof;
②d′ifor bolt feature image M1Direction vector d of ith pointiThe direction vector after being processed by the rotation matrix A is d'i＝Adi(ii) a Wherein A is a second order rotation matrix; and d'i＝
(u′i,v′i)TOf u's'iA horizontal component, v ', representing a rotated direction vector'iRepresents the vertical component thereof;
③eq+p'hardware feature representation image M0Adding the direction vector of the bolt characteristic rotation component p' to the upper matching point q, from e(x,y)＝(U(x,y),V(x,y))T、q＝(x,y)T、p′i＝(r′i,c′i) Three parts are composed of
By applying to the image I according to the above formula0Go through the image I0The s with the largest four positions represents that the four most similar bolts are detected, and the bolt identification is completed.
4. The high-precision operation method of the electrified overhaul robot facing the high-altitude settlement environment as claimed in claim 3, wherein in the bolt recognition and posture calculation step, the specific step of obtaining the posture of the hardware fitting plane under the camera coordinate system is as follows:
after the identification of the bolt is completed, coordinates P (X, Y) of four bolts in the color image are obtained, a depth value Z of the four bolts is extracted from the corresponding position of the depth image, then a 3D coordinate P (X, Y, Z) of the bolt under a camera coordinate system is obtained by using a formula (1-2),
in the above formula, f is the focal length of the camera, and dx and dy are the pixel sizes;
and calculating the space attitude of the bolt by adopting a universal least square method through the 3D coordinates P (X, Y, Z) of the bolt, and obtaining the attitude of the hardware fitting plane under a camera coordinate system, wherein the attitude comprises a yaw angle rz, a pitch angle ry and a roll angle rx.
5. The high-precision operation method of the charged overhaul robot facing the high-altitude settlement environment as claimed in claim 4, wherein the decomposition of the motion of the mechanical arm comprises the following steps of:
in order to align the hardware in the y-axis direction, the mechanical arm needs to be adjusted around the y-axis direction, and the steps are as follows: bolt tooling moves dx along the x direction1Advancing dz along the z direction1(ii) a Rotation angle ry of bolt tooling, wherein dx1，dz1Is represented as follows:
wherein, Depth1The distance between the bolt and the camera is represented, and the ry represents the pitch angle obtained by calculating the plane angle;
similarly, in order to align the hardware in the z-axis direction, the mechanical arm needs to move dx along the x direction first2Then proceeds dy along the y direction2(ii) a Finally, the bolt tool is rotated by an angle rz, wherein dx2，dy2Is represented as follows:
wherein,Depth2the distance from the bolt to the camera is represented, and rz represents a yaw angle obtained by calculating a plane angle; through the adjustment of the process, the bolt tool at the tail end of the mechanical arm is successfully aligned to the hardware fitting plane, so that the subsequent unloading work of the bolt can be conveniently completed.
Compared with the prior art, the invention has the beneficial effects that:
(1) compared with a monocular camera method, the method has the characteristics of small influence of ambient light, capability of directly acquiring spatial three-dimensional information and high precision;
(2) the edge shape characteristics of the hardware bolt in the belt direction are respectively extracted, then the similarity measurement algorithm is used for detecting and self-adaptive matching identification of the hardware bolt, and compared with a template matching algorithm based on gray scale, the method has the characteristics of no interference of target brightness and strong environmental light;
(3) compared with a method for calculating the posture of the bolt by extracting three-dimensional coordinates, the method has the advantages that the posture of the hardware bolt is calculated by extracting three-dimensional point clouds around the hardware bolt and using a least square method, so that the data volume is larger, and the precision is higher;
(4) the two key axes along the direction of the mechanical arm coordinate system are respectively subjected to motion decomposition, so that alignment and planar movement of the tool and the hardware fitting plane in a three-dimensional space are realized, and the method has the characteristics of direct and efficient effect;
(5) compared with other error compensation through a force feedback mode, the method has the advantages that the dominant mark is added, the settlement error is artificially compensated in a visual mode, and the method is low in cost and safer.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a schematic diagram of an RGB-D camera according to the present invention;
FIG. 3 is an RGB-D image of the hardware of the present invention;
FIG. 4 is an original image of hardware and bolt according to the present invention;
FIG. 5 is a characteristic image of hardware and bolt according to the present invention;
FIG. 6 is a diagram illustrating the positioning result of the hardware bolt of the present invention;
FIG. 7 is a schematic diagram of the robot arm unloading overhead hardware bolts and coordinates of the present invention;
FIG. 8 is an exploded view of the robotic arm movement of the present invention;
FIG. 9 is a schematic diagram of the error compensation based on manual monitoring according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The terms "first," "second," and the like, are used solely to distinguish one entity or action from another entity or action without necessarily being construed as indicating or implying any actual such relationship or order between such entities or actions.
As shown in fig. 1 to 9, in order to overcome the defects in the prior art, embodiments of the present application provide a high-precision operation method for an aerial settlement environment-oriented live-line maintenance robot.
In hardware, the robot comprises the following parts: (1) a double six-degree-of-freedom mechanical arm (comprising a main mechanical arm and an auxiliary mechanical arm); (2) an RGB-D camera (capable of outputting color image and depth image); (3) a bolt replacing tool (composed of a torque motor with the torque not less than 50 N.M and a sleeve, and capable of realizing the unloading and installation of hardware bolts) is adaptable to bolts with the sizes of M12-M16; (4) a 48V DC power supply; (5) bolt placement toolbox. The robot platform is lifted to the high altitude of more than 8 meters by a Z-shaped insulating bucket arm vehicle.
As shown in fig. 1, a high-precision operation method of an electrified overhaul robot facing an aerial settlement environment comprises the following specific steps:
s1, moving a robot to a working point, firstly, installing a robot platform at the tail end of an insulating bucket arm vehicle, then, slowly and remotely controlling the insulating bucket arm vehicle under the control and observation of a worker, and conveying the robot platform to the position 1000mm or so right in front of a target hardware fitting;
s2, acquiring an RGB-D image of the hardware, starting an RGB-D camera, directly acquiring a color image of the target hardware by the RGB camera, creating an infrared light spot of an area array on the surface of the target hardware by a structured light generator, acquiring left and right images by the binocular camera, performing rapid stereo matching on feature points of the left and right images, calculating a depth map of the target hardware according to a triangulation distance measuring principle, and finally acquiring the RGB-D image of the target hardware;
s3, bolt identification and posture calculation, wherein based on the hexagon characteristics of the bolts, the bolt identification is carried out by adopting a shape template matching algorithm, the spatial posture of the bolts is calculated by adopting a least square method through extracting a planar 3D point cloud in the middle of four bolts, and the posture of a hardware fitting plane under a camera coordinate system is obtained;
s4, the bolt tool is aligned to the hardware fitting, the movement of the mechanical arm is decomposed, and the bolt tool at the tail end of the mechanical arm is successfully aligned to the plane of the hardware fitting;
s5, based on error adjustment of manual monitoring, marking a circle mark on a picture, right opposite to the bolt, of the tool on the monitoring picture, fitting the bolt head, and performing position control on the mechanical arm according to the circle mark, so that settlement errors are compensated, and the center shaft of the bolt tool is aligned with the bolt head;
s6, carrying out hardware fitting overhauling operation, wherein the robot carries out overhauling operation on the hardware fitting after the central shaft of the bolt tool is aligned with the bolt head.
As shown in fig. 2, the RGB-D camera includes an RGB camera, a binocular camera composed of a group of infrared cameras, and a structural light generator, and a light supplement lamp is provided beside the camera, and the light supplement lamp is used for the RGB camera to directly obtain the color image of the target hardware under the dark light condition.
As shown in fig. 3, in the step of bolt recognition and posture calculation, the concrete steps of performing bolt recognition based on a shape template matching algorithm are as follows:
hardware image I in RGB-D image of target hardware0From the hardware image I0In acquiring bolt image I1As in fig. 4;
bolt image I1Carrying out edge feature extraction processing by Sobel and Canny operators and the like, and obtaining a processed image M1From a series of points pi＝(ri,ci)TAnd a corresponding direction vector di＝(ui,vi)TN, wherein:
i represents a bolt feature image M1I 1,2,.., n;
②riand ciRespectively representing bolt feature images M1The ith characteristic point is located in a row and a column;
③uiand viRespectively representing bolt feature images M1The ith characteristic point corresponds to the horizontal component and the vertical component of the direction vector;
t represents the matrix transposition;
transforming M using a second order standard rotation matrix A1And separating the translated portion to obtain a rotated component p'i(p′i＝Api) And a corresponding direction vector d'i(d′i＝Adi) From the hardware image I0After the acquisition processImage M of0(ii) a As shown in fig. 5.
Suppose that at M0One of the characteristic points q ═ x, y)TAnd the corresponding direction vector is e(x,y)＝(U(x,y),V(x,y))TWherein U is(x,y)Representing the horizontal component of the direction vector, V(x,y)The vertical component is represented. When bolt template image I1On hardware original image I0Q ═ a certain position (x, y)TWhen a match is made, the process can be converted to computing M1At M0The sum of the dot products of the normalized vectors at the previous point q (x, y), i.e., the similarity measure s, is given by the formula:
in the above formula:
s represents a bolt feature image M1Hardware fitting characteristic image M0Last point q ═ x, y)TSimilarity metric values in matching; x represents the abscissa of the matching position q point, and y represents the ordinate thereof;
②d′ifor bolt feature image M1Direction vector d of ith pointiThe direction vector after being processed by the rotation matrix A is d'i＝Adi(ii) a Wherein A is a second order rotation matrix; and d'i＝
(u′i,v′i)TOf u's'iA horizontal component, v ', representing a rotated direction vector'iRepresents the vertical component thereof;
③eq+p'hardware feature representation image M0Adding the direction vector of the bolt characteristic rotation component p' to the upper matching point q, from e(x,y)＝(U(x,y),V(x,y))T、q＝(x,y)T、p′i＝(r′i,c′i) Three parts are composed of
According to the formula, the hardware image I is processed0Go through the image I0The maximum similarity metric value s of the four positions in the hardware fitting image represents that the four most similar bolts are detected, and therefore the bolt identification in the hardware fitting image is completed. As shown in fig. 6, the similarity measure s has the following two features: (1) the method has adaptability to occlusion and image noise, and is characterized in that when an edge extraction operator calculates the image M1If some edge of the object is searching for the image M0If the signal is lost, the noise will cause random direction vectors, which will not affect the total size of s; (2) the similarity measure s is also invariant to any lighting variations, mainly because lighting of any brightness does not affect the gradient direction of the direction vector.
In the bolt identification and posture calculation steps, the specific steps of obtaining the posture of the hardware fitting plane under the camera coordinate system are as follows:
after the identification of the bolt is completed, coordinates P (X, Y) of four bolts in the color image are obtained, a depth value Z of the four bolts is extracted from the corresponding position of the depth image, then a 3D coordinate P (X, Y, Z) of the bolt under a camera coordinate system is obtained by using a formula (1-2),
in the above formula, f is the focal length of the camera, and dx and dy are the pixel sizes;
and calculating the space attitude of the bolt by adopting a universal least square method through the 3D coordinates P (X, Y, Z) of the bolt, and obtaining the attitude of the hardware fitting plane under a camera coordinate system, wherein the attitude comprises a yaw angle rz, a pitch angle ry and a roll angle rx.
When carrying out the uninstallation of gold utensil bolt and installation, the terminal bolt frock of arm needs to adjust well gold utensil plane, otherwise will appear the bolt and block, the uninstallation condition of failing. Therefore, the moving direction of the robot arm needs to be resolved along the hardware plane. According to the bolt attitude positioning method introduced in the steps, the angle of the hardware fitting plane under the camera coordinate system can be calculated and obtained, wherein the angle comprises rz, ry and rx, and rz and ry are most important for smoothly unloading the bolt. Fig. 7 is a schematic diagram and a coordinate schematic diagram of the robot arm for unloading the high-altitude hardware fitting bolt, and the motion of the robot arm is decomposed, including the decomposition around the y axis and the z axis of the robot arm as follows:
as shown in fig. 8 (a), in order to align the hardware in the y-axis direction, the robot arm needs to be adjusted around the y-axis direction, and the steps are as follows: bolt tooling moves dx along the x direction1Advancing dz along the z direction1(ii) a Rotation angle ry of bolt tooling, wherein dx1，dz1Is represented as follows:
wherein, Depth1The distance from the bolt to the camera is represented, and ry represents the pitch angle calculated from the plane angle.
Similarly, as shown in fig. 8 (b), in order to align the hardware in the z-axis direction, the robot arm needs to move dx along the x-direction first2Then proceeds dy along the y direction2(ii) a Finally, the bolt tool is rotated by an angle rz, wherein dx2，dy2Is represented as follows:
wherein, Depth2The distance from the bolt to the camera is indicated, and rz indicates the calculated yaw angle of the plane angle.
Through the adjustment of the process, the bolt tool at the tail end of the mechanical arm is successfully aligned to the hardware fitting plane, so that the subsequent unloading work of the bolt can be conveniently completed.
The error adjustment based on manual monitoring has the compensation effect as shown in fig. 9, and the manual error compensation amount can be recorded in the robot software system in time, so that the error compensation based on manual monitoring is not needed when other bolts are unloaded, and the efficiency of automatic operation is greatly improved.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (5)
1. A high-precision operation method of an electrified overhaul robot for an aerial settlement environment is characterized in that the robot comprises a mechanical arm, an RGB-D camera, a bolt replacing tool, a direct-current power supply and a bolt placing tool box, and the operation method comprises the following specific steps:
the robot moves to a working point, and the insulating bucket arm vehicle sends the robot to the position right in front of the target hardware fitting;
acquiring an RGB-D image of the hardware, starting an RGB-D camera, directly acquiring a color image of a target hardware by the RGB camera, creating an infrared light spot of an area array on the surface of the target hardware by a structured light generator, acquiring left and right images by a binocular camera, performing rapid stereo matching on feature points of the left and right images, calculating a depth image of the target hardware according to a triangulation distance measuring principle, and finally acquiring the RGB-D image of the target hardware;
the method comprises the steps of bolt identification and posture calculation, wherein based on the hexagon characteristics of bolts, bolt identification is carried out by adopting a shape template matching algorithm, the spatial posture of the bolts is calculated by adopting a least square method by extracting a planar 3D point cloud in the middle of four bolts, and the posture of a hardware fitting plane under a camera coordinate system is obtained;
the bolt tool is opposite to the hardware fitting, the movement of the mechanical arm is decomposed, and the bolt tool at the tail end of the mechanical arm is successfully aligned to the plane of the hardware fitting;
based on error adjustment of manual monitoring, a circle mark is marked on a picture, right opposite to the bolt, of the tool on a monitoring picture, the bolt head is fitted, and position control of the mechanical arm is performed according to the circle mark, so that settlement errors are compensated, and alignment of a central shaft of the bolt tool and the bolt head is realized;
and (4) carrying out maintenance operation on the hardware fitting after the central shaft of the bolt tool is aligned with the bolt head by the robot.
2. The high-precision operation method of the electrified overhaul robot facing the high-altitude settlement environment as claimed in claim 1, wherein in the hardware RGB-D image acquisition step, the RGB-D camera further comprises a light supplement lamp, and the light supplement lamp is used for directly acquiring the color image of the target hardware by the RGB camera under the dark light condition.
3. The high-precision operation method of the electrified overhaul robot facing the high-altitude settlement environment as claimed in claim 1, wherein in the bolt identification and posture calculation step, the concrete steps of adopting a shape template matching algorithm to identify the bolt are as follows:
hardware image I in RGB-D image of target hardware0From the hardware image I0In acquiring bolt image I1；
Bolt image I1Performing edge feature extraction processing to obtain processed image M1From a series of points pi＝(ri,ci)TAnd a corresponding direction vector di＝(ui,vi)TN, wherein:
i represents a bolt feature image M1I 1,2,.., n;
②riand ciRespectively representing bolt feature images M1The ith characteristic point is located in a row and a column;
③uiand viRespectively representing bolt feature images M1The ith characteristic point corresponds to the horizontal component and the vertical component of the direction vector;
t represents the matrix transposition;
transforming M using a second order standard rotation matrix A1And separating the translated portion to obtain a rotated component p'i(p′i＝Api) And a corresponding direction vector d'i(d′i＝Adi) From the hardware image I0Acquiring a processed image M0；
Suppose that at M0One point q ═ (x, y)TAnd the corresponding direction vector is e(x,y)＝(U(x,y),V(x,y))TWherein U is(x,y)Representing the horizontal component of the direction vector, V(x,y)Representing a vertical component, image I1And image I0At a certain point q ═ (x, y)TWhen matching, the sum of dot products of the normalized direction vectors is the similarity measure s, and the calculation formula is as follows:
in the above formula:
s represents a bolt feature image M1Hardware fitting characteristic image M0Last point q ═ x, y)TSimilarity metric values in matching; x represents the abscissa of the matching position q point, and y represents the ordinate thereof;
②d′ifor bolt feature image M1Direction vector d of ith pointiThe direction vector after being processed by the rotation matrix A is d'i＝Adi(ii) a Wherein A is a second order rotation matrix; and d'i＝(u′i,v′i)TOf u's'iA horizontal component, v ', representing a rotated direction vector'iRepresents the vertical component thereof;
③eq+p'hardware feature representation image M0Adding the direction vector of the bolt characteristic rotation component p' to the upper matching point q, from e(x,y)＝(U(x,y),V(x,y))T、q＝(x,y)T、p′i＝(r′i,c′i) Three parts are composed of
By applying to the image I according to the above formula0Go through the image I0The s with the largest four positions represents that the four most similar bolts are detected, and the bolt identification is completed.
4. The high-precision operation method of the electrified overhaul robot facing the high-altitude settlement environment as claimed in claim 3, wherein in the bolt recognition and posture calculation step, the specific step of obtaining the posture of the hardware fitting plane under the camera coordinate system is as follows:
after the identification of the bolt is completed, coordinates P (X, Y) of four bolts in the color image are obtained, a depth value Z of the four bolts is extracted from the corresponding position of the depth image, then a 3D coordinate P (X, Y, Z) of the bolt under a camera coordinate system is obtained by using a formula (1-2),
in the above formula, f is the focal length of the camera, and dx and dy are the pixel sizes;
and calculating the space attitude of the bolt by adopting a universal least square method through the 3D coordinates P (X, Y, Z) of the bolt, and obtaining the attitude of the hardware fitting plane under a camera coordinate system, wherein the attitude comprises a yaw angle rz, a pitch angle ry and a roll angle rx.
5. The high-precision operation method of the charged overhaul robot facing the high-altitude settlement environment as claimed in claim 4, wherein the decomposition of the motion of the mechanical arm comprises the following steps of:
in order to align the hardware in the y-axis direction, the mechanical arm needs to be adjusted around the y-axis direction, and the steps are as follows: bolt tooling moves dx along the x direction1Advancing dz along the z direction1(ii) a Rotation angle ry of bolt tooling, wherein dx1，dz1Is represented as follows:
wherein, Depth1The distance between the bolt and the camera is represented, and the ry represents the pitch angle obtained by calculating the plane angle;
similarly, in order to align the hardware in the z-axis direction, the mechanical arm needs to move dx along the x direction first2Then proceeds dy along the y direction2(ii) a Finally, the bolt tool is rotated by an angle rz, wherein dx2，dy2Is represented as follows:
wherein, Depth2The distance from the bolt to the camera is represented, and rz represents a yaw angle obtained by calculating a plane angle; through the adjustment of the process, the bolt tool at the tail end of the mechanical arm is successfully aligned to the hardware fitting plane, so that the subsequent unloading work of the bolt can be conveniently completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111452665.2A CN114240982B (en) | 2021-11-30 | 2021-11-30 | High-precision operation method of electrified overhaul robot facing high-altitude settlement environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111452665.2A CN114240982B (en) | 2021-11-30 | 2021-11-30 | High-precision operation method of electrified overhaul robot facing high-altitude settlement environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114240982A true CN114240982A (en) | 2022-03-25 |
| CN114240982B CN114240982B (en) | 2024-04-19 |
Family
ID=80752524
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111452665.2A Active CN114240982B (en) | 2021-11-30 | 2021-11-30 | High-precision operation method of electrified overhaul robot facing high-altitude settlement environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114240982B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032816A (en) * | 2019-04-19 | 2019-07-19 | 中科新松有限公司 | Mechanical arm torque evaluation method and estimating system |
| JP2021028100A (en) * | 2019-08-09 | 2021-02-25 | ＴｅｃｈＳｈａｒｅ株式会社 | Pickup robot, alignment system, and pickup method |
| CN113433129A (en) * | 2021-07-05 | 2021-09-24 | 无锡贝斯特精机股份有限公司 | Six-axis robot deburring cutter detection mechanism and method thereof |
-
2021
- 2021-11-30 CN CN202111452665.2A patent/CN114240982B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032816A (en) * | 2019-04-19 | 2019-07-19 | 中科新松有限公司 | Mechanical arm torque evaluation method and estimating system |
| JP2021028100A (en) * | 2019-08-09 | 2021-02-25 | ＴｅｃｈＳｈａｒｅ株式会社 | Pickup robot, alignment system, and pickup method |
| CN113433129A (en) * | 2021-07-05 | 2021-09-24 | 无锡贝斯特精机股份有限公司 | Six-axis robot deburring cutter detection mechanism and method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 刘志峰;王子涵;赵永胜;程强;: "一种开放环境下的螺栓位姿测量方法", 北京工业大学学报, no. 07, 10 July 2020 (2020-07-10) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114240982B (en) | 2024-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107590835B (en) | Mechanical arm tool quick-change visual positioning system and positioning method in nuclear environment | |
| CN106182004B (en) | The method of the industrial robot automatic pin hole assembly of view-based access control model guidance | |
| CN108182689B (en) | Three-dimensional identification and positioning method for plate-shaped workpiece applied to robot carrying and polishing field | |
| CN103264738B (en) | Automatic assembling system and method for vehicle windshield glass | |
| CN109341532B (en) | Automatic-assembly-oriented part coordinate calibration method based on structural features | |
| CN107030693B (en) | A kind of hot line robot method for tracking target based on binocular vision | |
| CN110293559B (en) | Installation method for automatically identifying, positioning and aligning | |
| CN110136211A (en) | A kind of workpiece localization method and system based on active binocular vision technology | |
| CN113103235B (en) | Method for vertically operating cabinet surface equipment based on RGB-D image | |
| CN113269723B (en) | Unordered grabbing system for parts with three-dimensional visual positioning and manipulator cooperative work | |
| CN109405755B (en) | Large-size pipe plate aperture and pitch measuring device and measuring method | |
| CN113246787A (en) | New energy automobile battery positioning method and device | |
| CN109648303A (en) | A kind of bus fitting screwed lock of hot line robot unloads equipment and its lock discharging method | |
| CN111452045A (en) | Reinforcing steel bar identification marking system and method based on stereoscopic vision | |
| CN115042175A (en) | Method for adjusting tail end posture of mechanical arm of robot | |
| CN106938463A (en) | A kind of method of large plate positioning crawl | |
| CN107328358B (en) | The measuring system and measurement method of aluminium cell pose | |
| CN109579698B (en) | Intelligent cargo detection system and detection method thereof | |
| CN111267094A (en) | Workpiece positioning and grabbing method based on binocular vision | |
| CN110568866A (en) | Three-dimensional curved surface vision guiding alignment system and alignment method | |
| CN207696530U (en) | A kind of robot for high-voltage hot-line work aerial work platform | |
| CN114240982B (en) | High-precision operation method of electrified overhaul robot facing high-altitude settlement environment | |
| WO2024035918A1 (en) | Autonomous solar installation using artificial intelligence | |
| CN114998422B (en) | High-precision rapid three-dimensional positioning system based on error compensation model | |
| CN115144879B (en) | Multi-machine multi-target dynamic positioning system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |