CN114037732A - Drainage wire identification, positioning and grabbing method - Google Patents

Drainage wire identification, positioning and grabbing method Download PDF

Info

Publication number
CN114037732A
CN114037732A CN202111238613.5A CN202111238613A CN114037732A CN 114037732 A CN114037732 A CN 114037732A CN 202111238613 A CN202111238613 A CN 202111238613A CN 114037732 A CN114037732 A CN 114037732A
Authority
CN
China
Prior art keywords
wire
drainage
camera
grabbing
exposed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111238613.5A
Other languages
Chinese (zh)
Inventor
林爽
刘冰倩
吴涵
黄建业
郑州
杨彦
廖飞龙
王仁书
罗翔
范元亮
辛宇晨
陈锦铭
陈海根
林桂明
陈永志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electric Power Research Institute of State Grid Fujian Electric Power Co Ltd
State Grid Fujian Electric Power Co Ltd
Yangtze Delta Region Institute of Tsinghua University Zhejiang
Original Assignee
Electric Power Research Institute of State Grid Fujian Electric Power Co Ltd
State Grid Fujian Electric Power Co Ltd
Yangtze Delta Region Institute of Tsinghua University Zhejiang
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electric Power Research Institute of State Grid Fujian Electric Power Co Ltd, State Grid Fujian Electric Power Co Ltd, Yangtze Delta Region Institute of Tsinghua University Zhejiang filed Critical Electric Power Research Institute of State Grid Fujian Electric Power Co Ltd
Priority to CN202111238613.5A priority Critical patent/CN114037732A/en
Publication of CN114037732A publication Critical patent/CN114037732A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a drainage wire identification, positioning and grabbing method which is used for accurately identifying a drainage wire through a camera at a mechanical arm in an automatic operation process of live-wire ignition operation, so that a gripper is positioned under the drainage wire, and an X axis or a Y axis of the gripper with the camera is parallel to an exposed wire part of the drainage wire to complete a grabbing task; method for dividing based on drainage wire example when positioning gripperIdentifying the exposed wire part, fitting the end point position of the exposed wire part, specifically, respectively translating the mechanical arm in the right direction, the lower direction, the left direction and the upper direction by a preset distance to draw a spatial virtual rectangular frame, establishing a user coordinate system by taking the rectangular frame plane as a coordinate plane, and setting the pixel coordinate of the exposed wire end point in four images as P1、P2、P3、P4Solving the motion problem of the hand grip point in the 3D-2D space coordinate system by using a PnP method; the automatic identification device is suitable for a 10kV live-wire work robot, and meets the requirements on accurate identification and grasping of the drainage wire in the automatic work process of live-wire work.

Description

Drainage wire identification, positioning and grabbing method
Technical Field
The invention relates to the technical field of power equipment, in particular to a drainage wire identification, positioning and grabbing method.
Background
With the continuous improvement of social requirements on the power supply safety and stability, the requirements on the operation quality of operators are higher and higher, and the operation quality and efficiency of the operators are difficult to improve again on the premise of ensuring the operation safety. At present, manual live working is performed by adopting a manual tool to perform operations such as stripping and cutting of cable sheaths, connection of drainage wires and the like. However, the power distribution network lines are usually very compact, the line-to-line distance is small, and short circuits are easily caused when operators are electrified to meet fire, so that accidents such as personal injuries and deaths are caused.
At present, the mechanical arm replaces manpower to complete distribution network live-line ignition operation in China, and the method has wide application prospect. However, in the operation process of the mechanical arm, a large number of application scenarios of visual auxiliary positioning are required, such as: identification of the lead, identification of the drainage wire, identification of the position of the fire, and the like. Wherein, because of the position of drainage wire and self gesture are unfixed, the visual identification of drainage wire and the accurate degree of difficulty of snatching are great. Therefore, a reliable algorithm for identifying, positioning and grabbing the drainage wire needs to be designed, is suitable for a 10kV live-wire work robot, and meets the requirements of accurate identification and grabbing of the drainage wire in the automatic work process of live-wire work.
Disclosure of Invention
The invention provides a drainage wire identification, positioning and grabbing method which is suitable for a 10kV live-wire work robot and meets the requirements of accurate identification and grabbing of a drainage wire in the automatic work process of live-wire work.
The invention adopts the following technical scheme.
A drainage wire identification, positioning and grabbing method is used for accurately identifying a drainage wire through a camera at a mechanical arm in an automatic operation process of live-wire ignition operation, and then grabbing an exposed wire part of the drainage wire by a gripper of the mechanical arm, wherein the gripper is positioned right below the drainage wire, and an X axis or a Y axis of the gripper with the camera is parallel to the exposed wire part of the drainage wire to complete a grabbing task; the method comprises the steps of identifying the exposed wire part based on drainage wire example segmentation when a gripper is positioned, fitting the end point position of the exposed wire part, specifically drawing a spatial virtual rectangular frame by controlling a mechanical arm to respectively translate a preset distance in the right direction, the lower direction, the left direction and the upper direction, establishing a user coordinate system by taking the rectangular frame plane as a coordinate plane, and establishing a user coordinate system by taking the pixel coordinate of the drainage wire end point in four images as P1、P2、P3、P4And solving the hand grip point motion problem in the 3D-2D space coordinate system by using a PnP method.
The method trains a multi-classification network using the SOLOV2 image segmentation technique for performing drainage wire instance segmentation to identify bare wires at the stripping and cutting locations.
The PnP method solves the motion problem of the end point of the drainage line, and specifically uses the EPnP method to solve the motion problem of the end point of the drainage line according to the mark point P1、P2、P3、P4Solving the pose change required by the camera to reach the drainage wire position according to the coordinate of the camera coordinate and the virtual coordinate plane mark point coordinate, wherein the process comprises the following steps:
and calculating the coordinates of the mark point under the reference system of the camera:
Figure BDA0003318461140000021
the coordinates of the 3D reference point under the camera reference frame are calculated:
Figure BDA0003318461140000022
computing
Figure BDA0003318461140000023
Center of gravity of
Figure BDA0003318461140000024
And matrix A:
Figure BDA0003318461140000025
Figure BDA0003318461140000026
calculating { pi}i＝1，...nCenter of gravity p ofOAnd matrix B
Figure BDA0003318461140000027
Figure BDA0003318461140000028
Calculating the height H of the camera
H＝BTA
Computing the SVD decomposition of H:
H＝U∑VT
calculating rotation R required by the pose change of the camera:
R＝UVT
calculating the translation t of the camera pose:
Figure BDA0003318461140000029
the exposed wire part of the drainage wire is positioned at the stripped and cut wire of the drainage wire suspension section, and the length of the exposed wire part is a known value; let the end point of the current lead be A and the exposed junction of the lead be B, so as to
Figure BDA0003318461140000031
Representing a space vector from B to A, wherein the camera and the manipulator gripper are horizontally arranged, when the manipulator gripper moves, the camera is enabled to reach the level with the exposed position of the lead through a posture transformation Y axis, and the camera moves through the translation transformation solved by an EPnP algorithm, so that the grabbing position of the drainage lead can be reached, and the specific calculation method of the posture transformation amount and the movement amount of the camera is as follows:
step A1, calculating
Figure BDA0003318461140000032
The included angle theta between the Y axis of the camera and the Y axis of the camera under a camera coordinate system is expressed by
Figure BDA0003318461140000033
Step A2, calculating and
Figure BDA0003318461140000034
rotation axis of Y axis
Figure BDA0003318461140000035
By
Figure BDA0003318461140000036
And
Figure BDA0003318461140000037
cross product gives
Let a ═ a1，a2，a3)，b＝(b1，b2，b3) Are respectively as
Figure BDA0003318461140000038
And the unit vector of the Y axis, the cross product is expressed as:
Figure BDA0003318461140000039
step A3, calculating rotation vector
Figure BDA00033184611400000310
Step A4, taking the exposed middle position of the drainage wire as a grabbing point, wherein the camera and the grabbing point have a conversion relationship:
Figure BDA00033184611400000311
the subsequent wire grabbing operation comprises the following steps:
a5, identifying a drainage wire target in an image by an over-deep learning target detection algorithm;
a6, calculating the position and orientation of the drainage wire relative to the camera according to a p3p algorithm;
step A7, calculating the space coordinates of the drainage wire under the robot coordinate system through the pose;
step A8, controlling the movement of a gripper of the robot mechanical arm to the drainage wire coordinate position to perform wire grabbing action;
during the wire grabbing operation, the hand grip needs to be prevented from contacting with the exposed wire part, the exposed wire part is identified by adopting a combined algorithm that an HIS space model is used for segmenting a background image, a morphological method is used for carrying out noise reduction treatment on the environment, the exposed wire part is segmented, a region growing method is used for continuously segmenting the exposed wire part, and fragments generated in the segmentation are removed from the segmented exposed wire by using a morphological corrosion method;
the combination algorithm comprises the following steps:
b1, selecting an initial growth pixel point A on a drainage wire foreground image shot by a camera, marking the initial growth pixel point as f (i, j), and in the A neighborhood, judging that the difference between the gray value of the point to be detected and the gray value of the growth point is 1 or 0 according to the growth criterion;
and step B2, after the first region growing, the gray values of f (i-1, j), f (i, j-1) and f (i, j +1) of the selected central points are all different from the gray value of the selected central points by 1, and therefore, the gray values are merged. After the second region growing, f (i +1, j) is merged;
step B3, after the third region grows, f (i +1, j-1) and f (i +2, j) are combined;
step B4, when no pixel point meeting the growth criterion exists, stopping growing the area on the target image;
step B5, using mathematical morphology corrosion to remove partial fragments generated during image segmentation, further optimizing image details and improving image quality;
the corrosion method comprises the following steps:
for target image X, set A is eroded using set B, defined as:
Figure BDA0003318461140000041
Figure BDA0003318461140000042
if Ba is contained in X, the point a is recorded, and the set of all points a satisfying the above condition is called the result of corrosion of X by B.
The identification formula of the bare wire is expressed as:
Figure BDA0003318461140000043
the mechanical arm respectively translates 5 centimeters in the right direction, the lower direction, the left direction and the upper direction to draw a virtual rectangular frame in space.
The drainage wire is a section of arc-shaped wire hung on two sides of the tower pole in live-line ignition operation, is in a suspension shape, and is exposed by a section of stripped and cut wire.
The robot is a robot arm of a live-line welding operation robot, when automatic operation of the live-line welding operation is carried out, the robot arm is guided to move manually, an image of the drainage wire is shot in a camera on the robot arm, then example segmentation is carried out manually, namely, one point of a stripping and cutting section of the drainage wire is determined by clicking on the image, a grabbing point of a belt leather section behind the stripping and cutting section needing to be grabbed by the robot arm is determined, the robot can accurately identify the whole drainage wire on the image by an algorithm, a background image is filtered, then the robot controls the moving position and the rotating direction of the robot arm, the space coordinate of a designated grabbing point position on the drainage wire is calculated according to the relationship between the obtained new position of the wire and the motion of the robot arm, and the robot arm is controlled to accurately grab the drainage wire.
The scheme of the invention is based on a drainage wire identification, positioning and grabbing algorithm, is suitable for a 10kV live-wire work robot, meets the requirements on accurate identification and grabbing of the drainage wire in the automatic work process of the live-wire work, can meet the operation and maintenance requirements of the live-wire work, saves the labor cost, improves the safety and reliability, and reduces the influence on other areas of the circuit.
Drawings
The invention is described in further detail below with reference to the following figures and detailed description:
FIG. 1 is a schematic view of a drainage wire;
FIG. 2 is a schematic view of a robotic arm gripper grasping a drainage wire;
in the figure: 1-a drainage wire; 2-exposing the lead part; 3-drainage line endpoint; 4. a mechanical arm; 5-camera.
Detailed Description
As shown in the figure, the method for identifying, positioning and grabbing the drainage wire is used for accurately identifying the drainage wire 1 through a camera 5 at a mechanical arm 4 in the automatic operation process of live-wire ignition operation and then grabbing the exposed wire part 2 of the drainage wire by a gripper of the mechanical arm, and the gripper is positioned right below the drainage wire and the X axis or the Y axis of the gripper with the camera is parallel to the exposed wire part of the drainage wire to complete a grabbing task; the method comprises the steps of identifying the exposed wire part based on drainage wire example segmentation when a gripper is positioned, fitting the end point position of the exposed wire part, specifically drawing a spatial virtual rectangular frame by controlling a mechanical arm to respectively translate a preset distance in the right direction, the lower direction, the left direction and the upper direction, establishing a user coordinate system by taking the rectangular frame plane as a coordinate plane, and establishing a user coordinate system by taking the pixel coordinate of the drainage wire end point in four images as P1、P2、P3、P4And solving the hand grip point motion problem in the 3D-2D space coordinate system by using a PnP method.
The method trains a multi-classification network using the SOLOV2 image segmentation technique for performing drainage wire instance segmentation to identify bare wires at the stripping and cutting locations.
The PnP method solves the motion problem of the drainage wire endpoint 3, specifically uses EPnP to solve the motion problem according to the mark point P1、P2、P3、P4Solving the pose change required by the camera to reach the drainage wire position according to the coordinate of the camera coordinate and the virtual coordinate plane mark point coordinate, wherein the process comprises the following steps:
and calculating the coordinates of the mark point under the reference system of the camera:
Figure BDA0003318461140000051
the coordinates of the 3D reference point under the camera reference frame are calculated:
Figure BDA0003318461140000052
computing
Figure BDA0003318461140000053
Center of gravity of
Figure BDA0003318461140000054
And matrix A:
Figure BDA0003318461140000061
Figure BDA0003318461140000062
calculating { pi}i＝1，...nCenter of gravity p ofOAnd matrix B
Figure BDA0003318461140000063
Figure BDA0003318461140000064
Calculating the height H of the camera
H＝BTA
Computing the SVD decomposition of H:
H＝U∑VT
calculating rotation R required by the pose change of the camera:
R＝UVT
calculating the translation t of the camera pose:
Figure BDA0003318461140000065
the exposed wire part of the drainage wire is positioned at the stripped and cut wire of the drainage wire suspension section, and the length of the exposed wire part is a known value; let the end point of the current lead be A and the exposed junction of the lead be B, so as to
Figure BDA0003318461140000066
Representing a space vector from B to A, wherein the camera and the manipulator gripper are horizontally arranged, when the manipulator gripper moves, the camera is enabled to reach the level with the exposed position of the lead through a posture transformation Y axis, and the camera moves through the translation transformation solved by an EPnP algorithm, so that the grabbing position of the drainage lead can be reached, and the specific calculation method of the posture transformation amount and the movement amount of the camera is as follows:
step A1, calculating
Figure BDA0003318461140000067
The included angle theta between the Y axis of the camera and the Y axis of the camera under a camera coordinate system is expressed by
Figure BDA0003318461140000068
Step A2, calculating and
Figure BDA0003318461140000069
rotation axis of Y axis
Figure BDA00033184611400000610
By
Figure BDA00033184611400000611
And
Figure BDA00033184611400000612
the cross product gives the value of (a)1,a2,a3),b＝(b1,b2,b3) Are respectively as
Figure BDA00033184611400000613
And the unit vector of the Y axis, the cross product is expressed as:
Figure BDA00033184611400000614
step A3, calculating rotation vector
Figure BDA0003318461140000071
Step A4, taking the exposed middle position of the drainage wire as a grabbing point, wherein the camera and the grabbing point have a conversion relationship:
Figure BDA0003318461140000072
the subsequent wire grabbing operation comprises the following steps:
a5, identifying a drainage wire target in an image by an over-deep learning target detection algorithm;
a6, calculating the position and orientation of the drainage wire relative to the camera according to a p3p algorithm;
step A7, calculating the space coordinates of the drainage wire under the robot coordinate system through the pose;
step A8, controlling the movement of a gripper of the robot mechanical arm to the drainage wire coordinate position to perform wire grabbing action;
during the wire grabbing operation, the hand grip needs to be prevented from contacting with the exposed wire part, the exposed wire part is identified by adopting a combined algorithm that an HIS space model is used for segmenting a background image, a morphological method is used for carrying out noise reduction treatment on the environment, the exposed wire part is segmented, a region growing method is used for continuously segmenting the exposed wire part, and fragments generated in the segmentation are removed from the segmented exposed wire by using a morphological corrosion method;
the combination algorithm comprises the following steps:
b1, selecting an initial growth pixel point A on a drainage wire foreground image shot by a camera, marking the initial growth pixel point as f (i, j), and in the A neighborhood, judging that the difference between the gray value of the point to be detected and the gray value of the growth point is 1 or 0 according to the growth criterion;
and step B2, after the first region growing, the gray values of f (i-1, j), f (i, j-1) and f (i, j +1) of the selected central points are all different from the gray value of the selected central points by 1, and therefore, the gray values are merged. After the second region growing, f (i +1, j) is merged;
step B3, after the third region grows, f (i +1, j-1) and f (i +2, j) are combined;
step B4, when no pixel point meeting the growth criterion exists, stopping growing the area on the target image;
step B5, using mathematical morphology corrosion to remove partial fragments generated during image segmentation, further optimizing image details and improving image quality;
the corrosion method comprises the following steps:
for target image X, set A is eroded using set B, defined as:
Figure BDA0003318461140000073
Figure BDA0003318461140000074
if Ba is contained in X, the point a is recorded, and the set of all points a satisfying the above condition is called the result of corrosion of X by B.
The identification formula of the bare wire is expressed as:
Figure BDA0003318461140000081
the mechanical arm respectively translates 5 centimeters in the right direction, the lower direction, the left direction and the upper direction to draw a virtual rectangular frame in space.
The drainage wire is a section of arc-shaped wire hung on two sides of the tower pole in live-line ignition operation, is in a suspension shape, and is exposed by a section of stripped and cut wire.
The robot is a robot arm of a live-line welding operation robot, when automatic operation of the live-line welding operation is carried out, the robot arm is guided to move manually, an image of the drainage wire is shot in a camera on the robot arm, then example segmentation is carried out manually, namely, one point of a stripping and cutting section of the drainage wire is determined by clicking on the image, a grabbing point of a belt leather section behind the stripping and cutting section needing to be grabbed by the robot arm is determined, the robot can accurately identify the whole drainage wire on the image by an algorithm, a background image is filtered, then the robot controls the moving position and the rotating direction of the robot arm, the space coordinate of a designated grabbing point position on the drainage wire is calculated according to the relationship between the obtained new position of the wire and the motion of the robot arm, and the robot arm is controlled to accurately grab the drainage wire.

Claims (7)

1. The utility model provides a drainage wire discernment location snatchs method for in the live working automation operation process, the camera through arm department is to the accurate discernment of drainage wire, and then snatchs its exposed wire position with the tongs of arm, its characterized in that: the method comprises the steps that a gripper is positioned right below a drainage wire, and the X axis or the Y axis of the gripper with the camera is parallel to the exposed wire part of the drainage wire so as to complete a gripping task; the method comprises the steps of identifying the exposed wire part based on drainage wire example segmentation when a gripper is positioned, fitting the end point position of the exposed wire part, specifically drawing a spatial virtual rectangular frame by controlling a mechanical arm to respectively translate a preset distance in the right direction, the lower direction, the left direction and the upper direction, establishing a user coordinate system by taking the rectangular frame plane as a coordinate plane, and establishing a user coordinate system by taking the pixel coordinate of the drainage wire end point in four images as P1、P2、P3、P4And solving the hand grip point motion problem in the 3D-2D space coordinate system by using a PnP method.
2. The method for identifying, positioning and grabbing the drainage wires according to claim 1, is characterized in that: the method trains a multi-classification network using the SOLOV2 image segmentation technique for performing drainage wire instance segmentation to identify bare wires at the stripping and cutting locations.
3. The method for identifying, positioning and grabbing the drainage wires according to claim 1, is characterized in that: the PnP method solves the motion problem of the end point of the drainage line, and specifically uses the EPnP method to solve the motion problem of the end point of the drainage line according to the mark point P1、P2、P3、P4Solving the pose change required by the camera to reach the drainage wire position according to the coordinate of the camera coordinate and the virtual coordinate plane mark point coordinate, wherein the process comprises the following steps:
and calculating the coordinates of the mark point under the reference system of the camera:
Figure FDA0003318461130000011
the coordinates of the 3D reference point under the camera reference frame are calculated:
Figure FDA0003318461130000012
computing
Figure FDA0003318461130000013
Center of gravity of
Figure FDA0003318461130000014
And matrix A:
Figure FDA0003318461130000015
Figure FDA0003318461130000016
calculating { pi}i＝1，...nCenter of gravity p ofOAnd matrix B
Figure FDA0003318461130000021
Figure FDA0003318461130000022
Calculating the height H of the camera
H＝BTA
Computing the SVD decomposition of H:
H＝U∑VT
calculating rotation R required by the pose change of the camera:
R＝UVT
calculating the translation t of the camera pose:
Figure FDA0003318461130000023
4. the method for identifying, positioning and grabbing the drainage wires according to claim 3, is characterized in that: the exposed wire part of the drainage wire is positioned at the stripped and cut wire of the drainage wire suspension section, and the length of the exposed wire part is a known value; let the end point of the current lead be A and the exposed junction of the lead be B, so as to
Figure FDA0003318461130000024
Representing a space vector from B to A, wherein the camera and the manipulator gripper are horizontally arranged, when the manipulator gripper moves, the camera is enabled to reach the level with the exposed position of the lead through a posture transformation Y axis, and the camera moves through the translation transformation solved by an EPnP algorithm, so that the grabbing position of the drainage lead can be reached, and the specific calculation method of the posture transformation amount and the movement amount of the camera is as follows:
step A1, calculating
Figure FDA0003318461130000025
The included angle theta between the Y axis of the camera and the Y axis of the camera under a camera coordinate system is expressed by
Figure FDA0003318461130000026
Step A2, calculating and
Figure FDA0003318461130000027
rotation axis of Y axis
Figure FDA0003318461130000028
By
Figure FDA0003318461130000029
And
Figure FDA00033184611300000210
cross product gives
Let a ═ a1，a2，a3)，b＝(b1，b2，b3) Are respectively as
Figure FDA00033184611300000211
And the unit vector of the Y axis, the cross product is expressed as:
Figure FDA00033184611300000212
step A3, calculating rotation vector
Figure FDA00033184611300000213
Step A4, taking the exposed middle position of the drainage wire as a grabbing point, wherein the camera and the grabbing point have a conversion relationship:
Figure FDA0003318461130000031
the subsequent wire grabbing operation comprises the following steps:
a5, identifying a drainage wire target in an image by an over-deep learning target detection algorithm;
a6, calculating the position and orientation of the drainage wire relative to the camera according to a p3p algorithm;
step A7, calculating the space coordinates of the drainage wire under the robot coordinate system through the pose;
step A8, controlling the movement of a gripper of the robot mechanical arm to the drainage wire coordinate position to perform wire grabbing action;
during the wire grabbing operation, the hand grip needs to be prevented from contacting with the exposed wire part, the exposed wire part is identified by adopting a combined algorithm that an HIS space model is used for segmenting a background image, a morphological method is used for carrying out noise reduction treatment on the environment, the exposed wire part is segmented, a region growing method is used for continuously segmenting the exposed wire part, and fragments generated in the segmentation are removed from the segmented exposed wire by using a morphological corrosion method;
the combination algorithm comprises the following steps:
b1, selecting an initial growth pixel point A on a drainage wire foreground image shot by a camera, marking the initial growth pixel point as f (i, j), and in the A neighborhood, judging that the difference between the gray value of the point to be detected and the gray value of the growth point is 1 or 0 according to the growth criterion;
and step B2, after the first region growing, the gray values of f (i-1, j), f (i, j-1) and f (i, j +1) of the selected central points are all different from the gray value of the selected central points by 1, and therefore, the gray values are merged. After the second region growing, f (i +1, j) is merged;
step B3, after the third region grows, f (i +1, j-1) and f (i +2, j) are combined;
step B4, when no pixel point meeting the growth criterion exists, stopping growing the area on the target image;
step B5, using mathematical morphology corrosion to remove partial fragments generated during image segmentation, further optimizing image details and improving image quality;
the corrosion method comprises the following steps:
for target image X, set A is eroded using set B, defined as:
Figure FDA0003318461130000032
Figure FDA0003318461130000033
if Ba is contained in X, the point a is recorded, and the set of all points a satisfying the above condition is called the result of corrosion of X by B.
The identification formula of the bare wire is expressed as:
Figure FDA0003318461130000034
5. the method for identifying, positioning and grabbing the drainage wires according to claim 1, is characterized in that: the mechanical arm respectively translates 5 centimeters in the right direction, the lower direction, the left direction and the upper direction to draw a virtual rectangular frame in space.
6. The method for identifying, positioning and grabbing the drainage wires according to claim 1, is characterized in that: the drainage wire is a section of arc-shaped wire hung on two sides of the tower pole in live-line ignition operation, is in a suspension shape, and is exposed by a section of stripped and cut wire.
7. The method for identifying, positioning and grabbing the drainage wires according to claim 1, is characterized in that: the robot is a robot arm of a live-line welding operation robot, when automatic operation of the live-line welding operation is carried out, the robot arm is guided to move manually, an image of the drainage wire is shot in a camera on the robot arm, then example segmentation is carried out manually, namely, one point of a stripping and cutting section of the drainage wire is determined by clicking on the image, a grabbing point of a belt leather section behind the stripping and cutting section needing to be grabbed by the robot arm is determined, the robot can accurately identify the whole drainage wire on the image by an algorithm, a background image is filtered, then the robot controls the moving position and the rotating direction of the robot arm, the space coordinate of a designated grabbing point position on the drainage wire is calculated according to the relationship between the obtained new position of the wire and the motion of the robot arm, and the robot arm is controlled to accurately grab the drainage wire.
CN202111238613.5A 2021-10-25 2021-10-25 Drainage wire identification, positioning and grabbing method Pending CN114037732A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111238613.5A CN114037732A (en) 2021-10-25 2021-10-25 Drainage wire identification, positioning and grabbing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111238613.5A CN114037732A (en) 2021-10-25 2021-10-25 Drainage wire identification, positioning and grabbing method

Publications (1)

Publication Number Publication Date
CN114037732A true CN114037732A (en) 2022-02-11

Family

ID=80141810

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111238613.5A Pending CN114037732A (en) 2021-10-25 2021-10-25 Drainage wire identification, positioning and grabbing method

Country Status (1)

Country Link
CN (1) CN114037732A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117249792A (en) * 2023-11-20 2023-12-19 国网浙江省电力有限公司杭州供电公司 Drainage wire length calculating device and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117249792A (en) * 2023-11-20 2023-12-19 国网浙江省电力有限公司杭州供电公司 Drainage wire length calculating device and method
CN117249792B (en) * 2023-11-20 2024-02-06 国网浙江省电力有限公司杭州供电公司 Drainage wire length calculating device and method

Similar Documents

Publication Publication Date Title
CN108280856B (en) Unknown object grabbing pose estimation method based on mixed information input network model
WO2018108098A1 (en) Autonomous operation method for live working robot based on multi-sensor information fusion
CN106044570B (en) It is a kind of that automatic identification equipment and method are hung using the coil of strip of machine vision
CN108182689B (en) Three-dimensional identification and positioning method for plate-shaped workpiece applied to robot carrying and polishing field
CN108247635B (en) Method for grabbing object by depth vision robot
WO2017015898A1 (en) Control system for robotic unstacking equipment and method for controlling robotic unstacking
CN111515945A (en) Control method, system and device for mechanical arm visual positioning sorting and grabbing
CN110605711B (en) Method, device and system for controlling cooperative robot to grab object
CN112845143A (en) Household garbage classification intelligent sorting system and method
CN113420746B (en) Robot visual sorting method and device, electronic equipment and storage medium
CN114037732A (en) Drainage wire identification, positioning and grabbing method
CN114029243B (en) Soft object grabbing and identifying method for sorting robot
CN114029951B (en) Robot autonomous recognition intelligent grabbing method based on depth camera
CN113911728A (en) Electric toothbrush brush head dynamic feeding system and feeding method based on vision
CN111300415B (en) Optimal operation position determination method used in live working robot scene
CN113538459A (en) Multi-mode grabbing obstacle avoidance detection optimization method based on drop point area detection
CN113715012B (en) Automatic assembling method and system for remote controller parts
CN113708290A (en) Live working robot drainage wire threading control method and device and robot terminal
CN114851209A (en) Industrial robot working path planning optimization method and system based on vision
CN113421283A (en) Automatic hoisting method and device for shield machine segments
CN115861780B (en) Robot arm detection grabbing method based on YOLO-GGCNN
JP5428921B2 (en) Work collection device and method from hanger line
Han et al. Toward fully automated metal recycling using computer vision and non-prehensile manipulation
JP2020041862A (en) Band-like object three-dimensional measurement method and band-like object three-dimensional measurement device
CN113814982A (en) Welding robot manipulator control method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination