CN111906767A - Vision rectification mechanical arm based on binocular structured light and rectification method - Google Patents

Vision rectification mechanical arm based on binocular structured light and rectification method Download PDF

Info

Publication number
CN111906767A
CN111906767A CN202010631576.3A CN202010631576A CN111906767A CN 111906767 A CN111906767 A CN 111906767A CN 202010631576 A CN202010631576 A CN 202010631576A CN 111906767 A CN111906767 A CN 111906767A
Authority
CN
China
Prior art keywords
mechanical arm
structured light
delta
gripper
piece
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010631576.3A
Other languages
Chinese (zh)
Inventor
王琪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Liaoran Vision Technology Co ltd
Original Assignee
Shenzhen Liaoran Vision Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Liaoran Vision Technology Co ltd filed Critical Shenzhen Liaoran Vision Technology Co ltd
Priority to CN202010631576.3A priority Critical patent/CN111906767A/en
Publication of CN111906767A publication Critical patent/CN111906767A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/10Programme-controlled manipulators characterised by positioning means for manipulator elements
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J15/00Gripping heads and other end effectors
    • B25J15/08Gripping heads and other end effectors having finger members
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J18/00Arms

Abstract

A vision deviation-rectifying mechanical arm based on binocular structured light and a deviation-rectifying method are characterized in that an industrial PC controls a mechanical arm controller to drive a mechanical arm to drive a gripper to realize positioning, meanwhile, a piece to be detected moved to a specific position is detected through binocular structured light equipment, the detection result is compared with an original storage position, and then deviation-rectifying processing is carried out, so that accurate control of the gripper is achieved, monitoring and deviation-rectifying operation of the specific position can be realized, and the vision deviation-rectifying mechanical arm is suitable for being used in occasions with low precision of operation positions, high randomness and high efficiency requirements.

Description

Vision rectification mechanical arm based on binocular structured light and rectification method
Technical Field
The invention relates to the field of mechanical automation, in particular to a binocular structured light-based vision deviation rectifying mechanical arm and a deviation rectifying method.
Background
The binocular structured light is an active vision field and has been applied to the fields of three-dimensional reconstruction, reverse engineering and the like. The binocular structured light is a multifunctional system which is integrated by a set of double vision systems and a set of structured light systems through calibration. The binocular stereo vision has the advantages of the binocular stereo vision in the aspect of object two-dimensional image matching, then depends on external light rays seriously in the three-dimensional reconstruction of the object, and has the defects of high pixel calculation matching difficulty, large calculation amount and the like.
As the structured light of active vision, the defect that binocular vision depends on light is just made up by independently emitting the light to establish the three-dimensional information of an object, and meanwhile, the structured light has the advantages of high speed, high precision and the like in the three-dimensional reconstruction process, and the requirement of people for extracting the information of each surface of an object is greatly met.
In industrial production, binocular vision is used for firstly acquiring two-dimensional information of an object and matching the two-dimensional information, and then structured light is used for carrying out three-dimensional reconstruction on the characteristic position of the object to obtain specific three-dimensional information of the characteristic. Greatly improving the precision of production and processing.
In the prior art, there is also a binocular structured light technology, for example, chinese patent document CN 208795188U describes a structured light binocular vision detection system, which is suitable for three-dimensional detection with different sizes and precision requirements, has high integration degree, can display the three-dimensional profile of the current detected object in real time, has strong timeliness and wide application prospect, but can position and correct the position when the moving object cannot be tracked.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a binocular structured light-based vision deviation rectifying mechanical arm and a deviation rectifying method, the system can judge the offset of a to-be-detected piece relative to the space three-dimensional coordinate of a grabbing point of a standard piece on a characteristic region, and the offset can be transmitted to a corresponding mechanical grabbing mechanism, so that the grabbing mechanism can complete accurate grabbing conveniently, and the whole deviation rectifying process of the system is realized.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a vision deviation rectifying mechanical arm based on binocular structured light comprises an industrial PC, wherein the industrial PC is connected with a mechanical arm controller through a bus, the mechanical arm controller controls a mechanical arm to position, a gripper is fixedly connected to the mechanical arm and is connected and controlled by a gripper controller, the gripper controller is connected with the industrial PC, the industrial PC is also connected with binocular structured light equipment, the industrial PC is connected with the binocular structured light equipment through Ethernet, a switch is required in the connection process due to the fact that the structured light is connected with three kilomega network cables, the industrial PC is connected with the gripper controller through a USB,
the gripper is rigidly connected with the tail end of the mechanical arm through the fixing piece, and the gripper can be controlled through controlling the mechanical arm.
The binocular structured light equipment is independent of the mechanical arm, supported by the corresponding mechanical supports and kept in a relative static state with the mechanical arm base all the time, and the binocular structured light equipment keeps a fixed relative position with the mechanical arm base after being fixed, so that the initial position cannot be changed due to movement of the mechanical arm when initial position calibration is carried out.
The bus between the industrial PC and the mechanical arm controller is a Profinet bus, and an industrial-grade universal bus can have higher response speed and optional spare part support.
The gripper is a positioning glue gun or a welding gun, and the corrected gripper can realize high-precision gluing or welding operation.
The deviation rectifying method using the mechanical arm is characterized by comprising the following specific steps of:
the method comprises the following steps: controlling the mechanical arm to drive the gripper to move and position to an initial position P point;
step two: acquiring images of the feature region of the dead standard part and corresponding three-dimensional cloud point information by using binocular structured light equipment, and calculating the spatial position coordinates of points to be grabbed in the feature region as (X0, Y0 and Z0);
step three: the industrial PC converts the coordinates (X0, Y0 and Z0) of the points to be grabbed into the grabbing positions of the grippers, controls the mechanical arm to drive the grippers to grab the workpiece, and returns to the initial position P0;
step four: after the piece to be detected moves to the fixed station, corresponding operations in the second step are repeated, namely space position coordinates (X1, Y1 and Z1) of the piece to be detected are obtained, deviation displacement (delta X, delta Y and delta Z) between the space position coordinates and (X0, Y0 and Z0) of the piece to be detected is calculated, if the deviation displacement (delta X, delta Y and delta Z) is within a deviation threshold range, the mechanical arm is controlled to drive the gripper to repeat the third step, if the deviation displacement (delta X, delta Y and delta Z) is not within the deviation threshold range, coordinates (X0, Y0 and Z0) of a point to be grabbed are changed into (X0+ delta X, Y0+ delta Y and Z0+ delta Z), and the third step is repeated;
step five: and when the piece to be detected exists, repeating the step four, and controlling the gripper to be positioned at the point P0 when the piece to be detected does not exist.
In the fourth step, Δ X, Δ Y, and Δ Z in the offset displacement (Δ X, Δ Y, and Δ Z) are vector values with positive and negative directions.
The detection steps of the binocular structured light device are as follows:
the method comprises the following steps: acquiring a two-dimensional image of the to-be-detected piece/the original standard piece by using a binocular camera;
step two: carrying out graying processing on the two-dimensional image, and then carrying out binarization processing;
step three: performing feature detection on the binarized image, and finding out corresponding grabbing points after detecting the features, namely two-dimensional coordinate (X, Y) values of the grabbing points;
step four: acquiring a grating image of the to-be-detected piece/the original standard piece by using a binocular camera and structured light;
step five: carrying out basic image processing on the obtained grating image;
step six: carrying out three-dimensional matching on grating images obtained by the left and right eye cameras;
step seven: mapping to a corresponding three-dimensional matching point cloud area according to the two-dimensional detection features obtained in the third step;
step eight: and finding out a Z-direction value corresponding to the grabbing point according to the two-dimensional coordinates (X and Y) of the grabbing point in the step three and the point cloud information of the characteristic area obtained in the step seven.
According to the vision deviation rectifying mechanical arm based on the binocular structured light and the deviation rectifying method, the mechanical arm is driven to drive the gripper to realize positioning through the industrial PC control mechanical arm controller, meanwhile, the piece to be detected moved to the specific position is detected through the binocular structured light equipment, the detection result is compared with the original storage position, deviation rectifying processing is carried out, accurate control of the gripper is achieved, monitoring and deviation rectifying operation of the specific position can be achieved, and the vision deviation rectifying mechanical arm based on the binocular structured light and the deviation rectifying method are suitable for being used in occasions with low operating position accuracy, high randomness and high efficiency requirements.
Drawings
The invention is further illustrated by the following examples in conjunction with the accompanying drawings:
FIG. 1 is a schematic diagram of the structure of the present invention;
FIG. 2 is a flow chart of the operation of the present invention;
FIG. 3 is a schematic diagram of the settlement process of spatial coordinates according to the present invention.
In the figure: the industrial PC1, the binocular structured light device 2, the arm controller 3, the arm 4, the gripper 5 and the gripper controller 6.
Detailed Description
As shown in fig. 1, the binocular structured light based vision correction mechanical arm comprises an industrial PC1, wherein an industrial PC1 is connected with a mechanical arm controller 3 through a bus, the mechanical arm controller 3 controls a mechanical arm 4 to be positioned, a gripper 5 is fixedly connected to the mechanical arm 4, the gripper 5 is connected and controlled by a gripper controller 6, the gripper controller 6 is connected with an industrial PC1, the industrial PC1 is further connected with a binocular structured light device 2, the industrial PC1 is connected with the binocular structured light device 2 through Ethernet, since the structured light is connected with three giga cables, a switch is required in the connection process, the industrial PC1 is connected with the gripper controller 6 through a USB,
the hand grip 5 is rigidly connected with the tail end of the mechanical arm 4 through a fixing piece, and the control of the hand grip 5 can be realized through controlling the mechanical arm 4.
The binocular structured light equipment 2 is independent of the mechanical arm 4, supported by corresponding mechanical supports and kept in a relative static state with the base of the mechanical arm 4 all the time, and the binocular structured light equipment 2 keeps a fixed relative position with the base of the mechanical arm 4 after being fixed, so that the initial position cannot be changed due to the movement of the mechanical arm 4 when the initial position calibration is carried out.
As shown in fig. 1, the above-described bus between the industrial PC1 and the robot controller 3 is a Profinet bus, and an industrial-grade general-purpose bus can have a faster response speed and optional spare part support.
The gripper 5 is a positioning glue gun or a welding gun, and the gripper 5 after deviation correction can realize high-precision gluing or welding operation.
As shown in fig. 2, the deviation rectifying method using the mechanical arm is characterized by comprising the following specific steps:
the method comprises the following steps: controlling the mechanical arm 4 to drive the gripper 5 to move and position to an initial position P0;
step two: the binocular structured light equipment 2 collects images of the feature region of the dead-proof standard piece and corresponding three-dimensional cloud point information, and spatial position coordinates of points to be grabbed in the feature region are calculated and recorded as X0, Y0 and Z0;
step three: the industrial PC1 converts the coordinates X0, Y0 and Z0 of points to be grabbed into the grabbing position of the hand grip 5, and controls the mechanical arm 4 to drive the hand grip 5 to grab the workpiece and then return to the initial position P0;
step four: after the piece to be detected moves to the fixed station, corresponding operations in the second step are repeated, namely spatial position coordinates X1, Y1 and Z1 of the piece to be detected are obtained, deviation displacements delta X, delta Y and delta Z between the piece to be detected and X0, Y0 and Z0 are calculated, if the deviation displacements delta X, delta Y and delta Z are within a deviation threshold range, the mechanical arm 4 is controlled to drive the gripper 5 to repeat the third step, and if the deviation displacements delta X, delta Y and delta Z are not within the deviation threshold range, coordinates X0, Y0 and Z0 of a point to be grabbed are changed into X0 plus delta X, Y0 plus Y, delta 0 plus delta Z, and the third step is repeated;
step five: when the object to be detected exists, the fourth step is repeated, and the gripper 5 is controlled to be positioned at the point P0 when the object to be detected does not exist.
In the fourth step, the deviation displacement Δ X, Δ Y, and Δ Z are vector values with positive and negative directions.
As shown in fig. 3, the detection steps of the binocular structured light device 2 are as follows:
the method comprises the following steps: acquiring a two-dimensional image of the to-be-detected piece/the original standard piece by using a binocular camera;
step two: carrying out graying processing on the two-dimensional image, and then carrying out binarization processing;
step three: performing feature detection on the binarized image, and finding out corresponding grabbing points after detecting the features, namely two-dimensional coordinate X and Y values of the grabbing points;
step four: acquiring a grating image of the to-be-detected piece/the original standard piece by using a binocular camera and structured light;
step five: carrying out basic image processing on the obtained grating image;
step six: carrying out three-dimensional matching on grating images obtained by the left and right eye cameras;
step seven: mapping to a corresponding three-dimensional matching point cloud area according to the two-dimensional detection features obtained in the third step;
step eight: and finding out a Z-direction value corresponding to the grabbing point according to the two-dimensional coordinates X and Y of the grabbing point in the step three and the point cloud information of the characteristic area obtained in the step seven.

Claims (7)

1. Visual deviation rectifying mechanical arm based on binocular structured light, which is characterized in that: the optical fiber cable clamp comprises an industrial PC (1), wherein the industrial PC (1) is connected with a mechanical arm controller (3) through a bus, the mechanical arm controller (3) controls a mechanical arm (4) to be positioned, a gripper (5) is fixedly connected onto the mechanical arm (4), the gripper (5) is connected and controlled by a gripper controller (6), the gripper controller (6) is connected with the industrial PC (1), and the industrial PC (1) is further connected with a binocular structured light device (2).
2. The binocular structured light-based vision correction mechanical arm is characterized in that the gripper (5) is rigidly connected with the tail end of the mechanical arm (4) through a fixing piece;
the binocular structured light equipment (2) is independent of the mechanical arm (4), supported by corresponding mechanical supports and kept in a relative static state with the base of the mechanical arm (4).
3. The binocular structured light-based vision correction mechanical arm as claimed in claim 1, wherein: and the bus between the industrial PC (1) and the mechanical arm controller (3) is a Profinet bus.
4. The binocular structured light-based vision correction mechanical arm as claimed in claim 2, wherein: the gripper (5) is a positioned glue gun or a positioned welding gun.
5. The deviation rectifying method using the mechanical arm of the claims 1 to 4 is characterized by comprising the following specific steps:
the method comprises the following steps: controlling the mechanical arm (4) to drive the gripper (5) to move and position to an initial position P0;
step two: the binocular structured light equipment (2) collects images of the characteristic region of the dead-proof standard piece and corresponding three-dimensional cloud point information, and spatial position coordinates of points to be grabbed in the characteristic region are calculated and recorded as (X0, Y0 and Z0);
step three: the industrial PC (1) converts the coordinates (X0, Y0 and Z0) of the point to be grabbed into the grabbing position of the grabbing hand (5), and controls the mechanical arm (4) to drive the grabbing hand (5) to grab the workpiece and then returns to the initial position P0;
step four: after the piece to be detected moves to the fixed station, corresponding operations in the second step are repeated, namely space position coordinates (X1, Y1 and Z1) of the piece to be detected are obtained, deviation displacement (delta X, delta Y and delta Z) between the space position coordinates and (X0, Y0 and Z0) of the piece to be detected is calculated, if the deviation displacement (delta X, delta Y and delta Z) is within a deviation threshold range, the mechanical arm (4) is controlled to drive the gripper (5) to repeat the third step, if the deviation displacement (delta X, delta Y and delta Z) is not within the deviation threshold range, coordinates (X0, Y0 and Z0) of a point to be grabbed are changed into (X0+ delta X, Y0+ delta Y and Z0+ Z), and the third step is repeated;
step five: and when the piece to be detected exists, repeating the step four, and controlling the gripper (5) to be positioned at the point P0 when the piece to be detected does not exist.
6. The deviation rectifying method according to claim 5, wherein: in the fourth step, in the deviation displacement (Δ X, Δ Y, Δ Z), Δ X, Δ Y, Δ Z are vector values with positive and negative directions.
7. The deviation rectifying method according to claim 5, wherein the binocular structured light device (2) is detected by the steps of:
the method comprises the following steps: acquiring a two-dimensional image of the to-be-detected piece/the original standard piece by using a binocular camera;
step two: carrying out graying processing on the two-dimensional image, and then carrying out binarization processing;
step three: performing feature detection on the binarized image, and finding out corresponding grabbing points after detecting the features, namely two-dimensional coordinate (X, Y) values of the grabbing points;
step four: acquiring a grating image of the to-be-detected piece/the original standard piece by using a binocular camera and structured light;
step five: carrying out basic image processing on the obtained grating image;
step six: carrying out three-dimensional matching on grating images obtained by the left and right eye cameras;
step seven: mapping to a corresponding three-dimensional matching point cloud area according to the two-dimensional detection features obtained in the third step;
step eight: and finding out a Z-direction value corresponding to the grabbing point according to the two-dimensional coordinates (X and Y) of the grabbing point in the step three and the point cloud information of the characteristic area obtained in the step seven.
CN202010631576.3A 2020-07-03 2020-07-03 Vision rectification mechanical arm based on binocular structured light and rectification method Pending CN111906767A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010631576.3A CN111906767A (en) 2020-07-03 2020-07-03 Vision rectification mechanical arm based on binocular structured light and rectification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010631576.3A CN111906767A (en) 2020-07-03 2020-07-03 Vision rectification mechanical arm based on binocular structured light and rectification method

Publications (1)

Publication Number Publication Date
CN111906767A true CN111906767A (en) 2020-11-10

Family

ID=73227255

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010631576.3A Pending CN111906767A (en) 2020-07-03 2020-07-03 Vision rectification mechanical arm based on binocular structured light and rectification method

Country Status (1)

Country Link
CN (1) CN111906767A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115167198A (en) * 2022-06-21 2022-10-11 沈阳新松机器人自动化股份有限公司 Wafer deviation rectifying system and method of double-end mechanical arm
CN116086359A (en) * 2023-04-07 2023-05-09 杭州键嘉医疗科技股份有限公司 Error measurement device and method for surgical instrument tracking array

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10315166A (en) * 1997-05-22 1998-12-02 Kawasaki Heavy Ind Ltd Remote visual display device provided with watching function
CN101486124A (en) * 2009-02-13 2009-07-22 南京工程学院 Multi-structured light binocular composite vision weld joint tracking method and device
CN103028816A (en) * 2012-12-24 2013-04-10 上海电机学院 Welding device and method based on binocular vision sensing of structured light
CN104942404A (en) * 2015-07-15 2015-09-30 广东工业大学 Dual-wavelength binocular vision seam tracking method and tracking system
CN107767423A (en) * 2017-10-10 2018-03-06 大连理工大学 A kind of mechanical arm target positioning grasping means based on binocular vision
CN107907048A (en) * 2017-06-30 2018-04-13 长沙湘计海盾科技有限公司 A kind of binocular stereo vision method for three-dimensional measurement based on line-structured light scanning

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10315166A (en) * 1997-05-22 1998-12-02 Kawasaki Heavy Ind Ltd Remote visual display device provided with watching function
CN101486124A (en) * 2009-02-13 2009-07-22 南京工程学院 Multi-structured light binocular composite vision weld joint tracking method and device
CN103028816A (en) * 2012-12-24 2013-04-10 上海电机学院 Welding device and method based on binocular vision sensing of structured light
CN104942404A (en) * 2015-07-15 2015-09-30 广东工业大学 Dual-wavelength binocular vision seam tracking method and tracking system
CN107907048A (en) * 2017-06-30 2018-04-13 长沙湘计海盾科技有限公司 A kind of binocular stereo vision method for three-dimensional measurement based on line-structured light scanning
CN107767423A (en) * 2017-10-10 2018-03-06 大连理工大学 A kind of mechanical arm target positioning grasping means based on binocular vision

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
梁晋等: "《3D反求技术》", 31 January 2019, 华中科技大学出版社 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115167198A (en) * 2022-06-21 2022-10-11 沈阳新松机器人自动化股份有限公司 Wafer deviation rectifying system and method of double-end mechanical arm
CN116086359A (en) * 2023-04-07 2023-05-09 杭州键嘉医疗科技股份有限公司 Error measurement device and method for surgical instrument tracking array

Similar Documents

Publication Publication Date Title
CN111775154B (en) Robot vision system
CN108182689B (en) Three-dimensional identification and positioning method for plate-shaped workpiece applied to robot carrying and polishing field
CN109794963B (en) Robot rapid positioning method facing curved surface component
CN102825602B (en) PSD (Position Sensitive Detector)-based industrial robot self-calibration method and device
CN108098762A (en) A kind of robotic positioning device and method based on novel visual guiding
CN102922521A (en) Mechanical arm system based on stereo visual serving and real-time calibrating method thereof
CN108942918B (en) Stereo positioning method based on line structured light
CN109291048B (en) Real-time online programming system and method for grinding and polishing industrial robot
CN113379849B (en) Robot autonomous recognition intelligent grabbing method and system based on depth camera
CN112917513A (en) TCP calibration method of three-dimensional dispensing needle head based on machine vision
CN111906767A (en) Vision rectification mechanical arm based on binocular structured light and rectification method
CN112648934B (en) Automatic elbow geometric form detection method
CN112643207B (en) Laser automatic derusting system and method based on computer vision
CN109465829A (en) A kind of industrial robot geometric parameter discrimination method based on transition matrix error model
CN111531407A (en) Workpiece attitude rapid measurement method based on image processing
CN110568866B (en) Three-dimensional curved surface visual guidance alignment system and alignment method
CN111267094A (en) Workpiece positioning and grabbing method based on binocular vision
CN110962127B (en) Auxiliary calibration device for tail end pose of mechanical arm and calibration method thereof
CN115042175A (en) Method for adjusting tail end posture of mechanical arm of robot
CN112958960B (en) Robot hand-eye calibration device based on optical target
CN110253582A (en) A kind of method of the mobile contraposition of manipulator on assembly line
CN215970736U (en) Steel rail marking device based on three-dimensional visual guidance
CN112123329A (en) Robot 3D vision hand-eye calibration method
CN115609586A (en) Robot high-precision assembling method based on grabbing pose constraint
CN212146434U (en) Simple and convenient robot hand-eye calibration system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20201110

WD01 Invention patent application deemed withdrawn after publication