CN110936369A - Binocular vision and mechanical arm based large-scale workpiece pose accurate measurement and grabbing device and method - Google Patents
Binocular vision and mechanical arm based large-scale workpiece pose accurate measurement and grabbing device and method Download PDFInfo
- Publication number
- CN110936369A CN110936369A CN201811114132.1A CN201811114132A CN110936369A CN 110936369 A CN110936369 A CN 110936369A CN 201811114132 A CN201811114132 A CN 201811114132A CN 110936369 A CN110936369 A CN 110936369A
- Authority
- CN
- China
- Prior art keywords
- pose
- mechanical arm
- workpiece
- binocular vision
- grabbing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a binocular vision and mechanical arm based device and a method for accurately measuring and grabbing the pose of a large-sized workpiece. The industrial mechanical arm is installed on the mechanical arm base, and the height of the mechanical arm base is designed according to task requirements. The binocular vision system is installed at the tail end of the mechanical arm, and the direction of the optical axis of a camera of the vision system is perpendicular to the Z axis of a tool coordinate system at the tail end of the mechanical arm. The end gripper is mounted to the end of the arm behind the binocular vision system along the Z-axis direction of the tool coordinate system. According to the invention, the binocular vision system and the mechanical arm are utilized to realize accurate pose measurement and stable grabbing of the heavy workpiece, and each work site is not required to teach grabbing of the heavy workpiece, so that a large amount of teaching workload and time are saved, the difficulty and requirement of mechanical positioning design are greatly reduced, and the flexibility and stability of the whole assembly system are improved.
Description
Technical Field
The invention relates to the field of robot measurement and control, in particular to a binocular vision and mechanical arm based large-scale workpiece pose accurate measurement and grabbing device and method.
Background
The existing robot vision servo technology is generally monocular vision, is mainly used for 2D and 2.5D assembly operation, and is generally used for grabbing and assembling small parts. First, the position of the parts to be assembled in a plane is measured using monocular vision. And then, controlling the mechanical arm and the gripper to grab the part to be assembled to finish the assembling operation. Because monocular vision is not sensitive to depth information, the monocular vision is only used for measuring displacement information of a plane and rotation information around a vertical plane axis, and the problem of pose measurement and grabbing of a large workpiece in a three-dimensional space is difficult to solve by the current robot vision servo technology.
Disclosure of Invention
In order to solve the problem of measuring and grabbing the three-dimensional space pose of a large workpiece, the invention aims to provide a device and a method for accurately measuring and grabbing the pose of the large workpiece based on binocular vision and a mechanical arm.
A binocular vision and mechanical arm based large-scale workpiece pose accurate measurement and grabbing device and method are characterized in that the device comprises: arm base, industrial robot arm, binocular vision, blue light area source, the industrial computer, terminal holder, the solenoid valve connecting wire, power gas circuit, the camera connecting wire, the light source connecting wire, its characterized in that, industrial robot arm installs on the arm base, and binocular vision installs at the arm end, and terminal holder is installed at the arm end, makes binocular vision be located between arm end and the holder.
Preferably, the visual field direction of the binocular vision is 90 degrees with the grabbing direction of the tail end clamp holder, and the structure enables the tail end clamp holder to be designed without considering the installation and arrangement problems of a visual system, so that the clamp holder for grabbing heavy workpieces can be designed conveniently.
Preferably, the binocular vision uses a large area light source, and the binocular camera acquires a visual field from an opening of the area light source.
Preferably, the end gripper is powered by an air circuit, and the release and the gripping of the clamping jaws are controlled by a solenoid valve.
Further, the method comprises two processes, offline and online:
an off-line process:
step S1: and putting the workpiece to be grabbed at any pose as a teaching pose. Manually demonstrating the grabbing pose TG of the mechanical arm under the pose of the workpiece;
step S2: the mechanical arm drives the binocular vision to move to a photographing pose TC, so that the coarse matching feature and the fine positioning feature of the workpiece to be grabbed are in the visual field of the binocular vision;
step S3: determining the pose TM of the workpiece according to the fine positioning feature points reconstructed by binocular vision, and calculating the relative pose TMG of the mechanical arm and the workpiece under the grabbing attitude according to the coordinate transformation relation, wherein the relative pose TMG is kept unchanged during grabbing of the workpiece;
an online process:
step S4: the mechanical arm drives the binocular vision to move to a photographing pose TC 'above the workpiece, the binocular vision system collects real-time images of the workpiece, coarse matching positioning and fine positioning are carried out, and the current pose TM' of the workpiece is determined.
Step S5: and according to the relative pose TMG of the mechanical arm and the workpiece in the grabbing pose obtained in the off-line process, obtaining the grabbing pose TG' of the mechanical arm in the current workpiece pose, sending an instruction to the mechanical arm, and enabling the mechanical arm to move to the grabbing pose to grab the workpiece.
According to the invention, the mechanical arm does not need to be taught for multiple times, and the workpiece placed in any posture can be accurately grasped by demonstrating only one grasping pose; the invention adopts a method combining the rough positioning of workpiece template matching and the precise positioning of geometric feature fitting, thereby not only improving the stability of workpiece positioning, but also measuring the pose of the workpiece with high precision and realizing high-precision grabbing operation.
Experimental production proves that the invention can realize accurate pose measurement and grabbing operation of large-size heavy workpieces, and greatly reduce teaching work of putting workpieces at different poses.
Drawings
FIG. 1 is a schematic diagram of a binocular vision and mechanical arm based large-scale workpiece pose measuring and grabbing device.
FIG. 2 is a flow chart of a method for measuring and grabbing the pose of a large workpiece based on binocular vision and a mechanical arm.
Wherein: the method comprises the following steps of 1-mechanical arm base, 2-industrial mechanical arm, 3-binocular vision, 4-blue light area light source, 5-terminal clamp, 6-electromagnetic valve connecting line, 7-power gas circuit, 8-camera connecting line, 9-light source connecting line, 10-industrial personal computer and 11-mechanical arm communication connecting line.
Detailed Description
In order to further understand the present invention, the following detailed description will be made with reference to the following examples, which are only used for explaining the present invention and are not to be construed as limiting the scope of the present invention.
The invention provides a binocular vision and mechanical arm based device and method for accurately measuring and grabbing the pose of a large workpiece, which are characterized by comprising the following steps: the industrial mechanical arm is characterized in that the industrial mechanical arm 2 is arranged on the mechanical arm base 1, the binocular vision 3 is arranged at the tail end of the mechanical arm, the tail end clamp 5 is arranged at the tail end of the mechanical arm, the binocular vision 3 is arranged between the tail end of the mechanical arm and the clamp, the view direction of the binocular vision 3 and the grabbing direction of the tail end clamp form 90 degrees, the structure enables the tail end clamp to be designed without considering the installation and arrangement problems of a vision system, the clamp for grabbing heavy workpieces is convenient to design, the binocular vision 3 uses a large-area surface light source, the binocular camera obtains the view from an opening of the surface light source, and the tail end clamp 5 is powered by the air path, the electromagnetic valve controls the release and the grasp of the clamping jaws.
The method comprises an off-line process and an on-line process:
an off-line process:
step S1: and putting the workpiece to be grabbed at any pose as a teaching pose. Manually demonstrating the grabbing pose TG of the mechanical arm under the pose of the workpiece;
step S2: the mechanical arm drives the binocular vision to move to a photographing pose TC, so that the coarse matching feature and the fine positioning feature of the workpiece to be grabbed are in the visual field of the binocular vision;
step S3: determining the pose TM of the workpiece according to the fine positioning feature points reconstructed by binocular vision, and calculating the relative pose TMG of the mechanical arm and the workpiece under the grabbing attitude according to the coordinate transformation relation, wherein the relative pose TMG is kept unchanged during grabbing of the workpiece;
an online process:
step S4: the mechanical arm drives the binocular vision to move to a photographing pose TC 'above the workpiece, the binocular vision system collects real-time images of the workpiece, coarse matching positioning and fine positioning are carried out, and the current pose TM' of the workpiece is determined.
Step S5: and according to the relative pose TMG of the mechanical arm and the workpiece in the grabbing pose obtained in the off-line process, obtaining the grabbing pose TG' of the mechanical arm in the current workpiece pose, sending an instruction to the mechanical arm, and enabling the mechanical arm to move to the grabbing pose to grab the workpiece.
According to the invention, the mechanical arm does not need to be taught for multiple times, and the workpiece placed in any posture can be accurately grasped by demonstrating only one grasping pose; the invention adopts a method combining the rough positioning of workpiece template matching and the precise positioning of geometric feature fitting, thereby not only improving the stability of workpiece positioning, but also measuring the pose of the workpiece with high precision and realizing high-precision grabbing operation.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
The above-described embodiments are merely illustrative of the embodiments of the present invention, and do not limit the spirit and scope of the present invention. Various modifications and improvements of the technical solutions of the present invention may be made by those skilled in the art without departing from the design concept of the present invention, and all of them should fall into the protection scope of the present invention.
Claims (5)
1. A binocular vision and mechanical arm based large-scale workpiece pose accurate measurement and grabbing device and method are characterized in that the device comprises: arm base (1), industrial machinery arm (2), binocular vision (3), blue light area source (5), industrial computer (10), terminal holder (5), solenoid valve connecting wire (6), power gas circuit (7), camera connecting wire (8), light source connecting wire (9), its characterized in that, industrial machinery arm (2) is installed on arm base (1), and binocular vision (3) are installed at the arm end, and terminal holder (5) are installed at the arm end, make binocular vision (3) be located between arm end and holder.
2. The device according to claim 1, characterized in that the view direction of the binocular vision (3) is 90 ° to the gripping direction of the end gripper, the structure being such that the end gripper (5) is designed without having to consider the mounting arrangement of the vision system, facilitating the design of the end gripper (5) for gripping heavy workpieces.
3. The device according to claim 1, characterized in that the binocular vision (3) uses a large area light source, and the binocular camera takes a view from an opening of the area light source.
4. Device according to claim 1, characterized in that the tip gripper (5) is powered by pneumatic circuits, the release and gripping of the jaws being controlled by solenoid valves.
5. The apparatus of claim 1, wherein the method comprises both offline and online processes:
an off-line process:
step S1: and putting the workpiece to be grabbed at any pose as a teaching pose. Manually demonstrating the grabbing pose TG of the mechanical arm under the pose of the workpiece;
step S2: the mechanical arm drives the binocular vision to move to a photographing pose TC, so that the coarse matching feature and the fine positioning feature of the workpiece to be grabbed are in the visual field of the binocular vision;
step S3: determining the pose TM of the workpiece according to the fine positioning feature points reconstructed by binocular vision, and calculating the relative pose TMG of the mechanical arm and the workpiece under the grabbing attitude according to the coordinate transformation relation, wherein the relative pose TMG is kept unchanged during grabbing of the workpiece;
an online process:
step S4: the mechanical arm drives the binocular vision to move to a photographing pose TC 'above the workpiece, the binocular vision system collects real-time images of the workpiece, coarse matching positioning and fine positioning are carried out, and the current pose TM' of the workpiece is determined.
Step S5: and according to the relative pose TMG of the mechanical arm and the workpiece in the grabbing pose obtained in the off-line process, obtaining the grabbing pose TG' of the mechanical arm in the current workpiece pose, sending an instruction to the mechanical arm, and enabling the mechanical arm to move to the grabbing pose to grab the workpiece.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811114132.1A CN110936369B (en) | 2018-09-25 | 2018-09-25 | Large-scale workpiece pose accurate measurement and grabbing method based on binocular vision and mechanical arm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811114132.1A CN110936369B (en) | 2018-09-25 | 2018-09-25 | Large-scale workpiece pose accurate measurement and grabbing method based on binocular vision and mechanical arm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110936369A true CN110936369A (en) | 2020-03-31 |
| CN110936369B CN110936369B (en) | 2022-10-04 |
Family
ID=69905123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811114132.1A Active CN110936369B (en) | 2018-09-25 | 2018-09-25 | Large-scale workpiece pose accurate measurement and grabbing method based on binocular vision and mechanical arm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110936369B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111751136A (en) * | 2020-06-29 | 2020-10-09 | 伯肯森自动化技术（上海）有限公司 | POS machine test system based on binocular vision subassembly |
| CN111993420A (en) * | 2020-08-10 | 2020-11-27 | 广州瑞松北斗汽车装备有限公司 | Fixed binocular vision 3D guide piece feeding system |
| CN112894823A (en) * | 2021-02-08 | 2021-06-04 | 珞石（山东）智能科技有限公司 | Robot high-precision assembling method based on visual servo |
| CN113093356A (en) * | 2021-03-18 | 2021-07-09 | 北京空间机电研究所 | Large-scale block optical component assembling method based on mechanical arm |
| CN113593050A (en) * | 2021-08-02 | 2021-11-02 | 南京南戈特机电科技有限公司 | Binocular vision guided robot intelligent assembly method, system and device |
| CN114248086A (en) * | 2021-12-30 | 2022-03-29 | 广东工业大学 | A flexible three-dimensional vision-guided robot alignment system and method |
| CN114952776A (en) * | 2022-06-16 | 2022-08-30 | 江西省智能产业技术创新研究院 | Composite robot suitable for multidimensional operation |
| CN116833724A (en) * | 2023-08-07 | 2023-10-03 | 中国铁建电气化局集团北方工程有限公司 | H-shaped steel column assembly visual guiding method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103203526A (en) * | 2012-01-16 | 2013-07-17 | 成都熊谷加世电器有限公司 | Laser visual tracking system |
| CN104325268A (en) * | 2014-11-04 | 2015-02-04 | 南京赫曼机器人自动化有限公司 | Industrial robot three-dimensional space independent assembly method based on intelligent learning |
| CN104634279A (en) * | 2013-11-13 | 2015-05-20 | 中国科学院沈阳计算技术研究所有限公司 | Vision-based automatic aviation oil mist nozzle atomization angle detection device and method |
| CN105856262A (en) * | 2016-05-16 | 2016-08-17 | 清华大学 | Method for detecting and recognizing object by small robot through touch sense |
| CN105965519A (en) * | 2016-06-22 | 2016-09-28 | 江南大学 | Vision-guided discharging positioning method of clutch |
| CN106778790A (en) * | 2017-02-15 | 2017-05-31 | 苏州博众精工科技有限公司 | A kind of target identification based on three-dimensional point cloud and localization method and system |
| CN107192331A (en) * | 2017-06-20 | 2017-09-22 | 佛山市南海区广工大数控装备协同创新研究院 | A kind of workpiece grabbing method based on binocular vision |
| CN107363403A (en) * | 2017-09-09 | 2017-11-21 | 深圳市华天世纪激光科技有限公司 | A kind of laser welding system based on CCD vision monitorings |
| CN107433573A (en) * | 2017-09-04 | 2017-12-05 | 上海理工大学 | Intelligent binocular captures mechanical arm automatically |
| CN108177150A (en) * | 2016-12-08 | 2018-06-19 | 广州映博智能科技有限公司 | Door of elevator positioning and grabbing device and the method for view-based access control model |
-
2018
- 2018-09-25 CN CN201811114132.1A patent/CN110936369B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103203526A (en) * | 2012-01-16 | 2013-07-17 | 成都熊谷加世电器有限公司 | Laser visual tracking system |
| CN104634279A (en) * | 2013-11-13 | 2015-05-20 | 中国科学院沈阳计算技术研究所有限公司 | Vision-based automatic aviation oil mist nozzle atomization angle detection device and method |
| CN104325268A (en) * | 2014-11-04 | 2015-02-04 | 南京赫曼机器人自动化有限公司 | Industrial robot three-dimensional space independent assembly method based on intelligent learning |
| CN105856262A (en) * | 2016-05-16 | 2016-08-17 | 清华大学 | Method for detecting and recognizing object by small robot through touch sense |
| CN105965519A (en) * | 2016-06-22 | 2016-09-28 | 江南大学 | Vision-guided discharging positioning method of clutch |
| CN108177150A (en) * | 2016-12-08 | 2018-06-19 | 广州映博智能科技有限公司 | Door of elevator positioning and grabbing device and the method for view-based access control model |
| CN106778790A (en) * | 2017-02-15 | 2017-05-31 | 苏州博众精工科技有限公司 | A kind of target identification based on three-dimensional point cloud and localization method and system |
| CN107192331A (en) * | 2017-06-20 | 2017-09-22 | 佛山市南海区广工大数控装备协同创新研究院 | A kind of workpiece grabbing method based on binocular vision |
| CN107433573A (en) * | 2017-09-04 | 2017-12-05 | 上海理工大学 | Intelligent binocular captures mechanical arm automatically |
| CN107363403A (en) * | 2017-09-09 | 2017-11-21 | 深圳市华天世纪激光科技有限公司 | A kind of laser welding system based on CCD vision monitorings |
Non-Patent Citations (2)
| Title |
|---|
| 全燕鸣等: "基于双目视觉的工件尺寸在机三维测量", 《光学精密工程》 * |
| 金路等: "一种基于工业机器人的三维视觉检测系统", 《微型电脑应用》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111751136A (en) * | 2020-06-29 | 2020-10-09 | 伯肯森自动化技术（上海）有限公司 | POS machine test system based on binocular vision subassembly |
| CN111993420A (en) * | 2020-08-10 | 2020-11-27 | 广州瑞松北斗汽车装备有限公司 | Fixed binocular vision 3D guide piece feeding system |
| CN112894823A (en) * | 2021-02-08 | 2021-06-04 | 珞石（山东）智能科技有限公司 | Robot high-precision assembling method based on visual servo |
| CN112894823B (en) * | 2021-02-08 | 2022-06-21 | 珞石（山东）智能科技有限公司 | Robot high-precision assembling method based on visual servo |
| CN113093356A (en) * | 2021-03-18 | 2021-07-09 | 北京空间机电研究所 | Large-scale block optical component assembling method based on mechanical arm |
| CN113093356B (en) * | 2021-03-18 | 2022-08-12 | 北京空间机电研究所 | A robotic arm-based assembly method for large-scale segmented optical components |
| CN113593050A (en) * | 2021-08-02 | 2021-11-02 | 南京南戈特机电科技有限公司 | Binocular vision guided robot intelligent assembly method, system and device |
| CN114248086A (en) * | 2021-12-30 | 2022-03-29 | 广东工业大学 | A flexible three-dimensional vision-guided robot alignment system and method |
| CN114952776A (en) * | 2022-06-16 | 2022-08-30 | 江西省智能产业技术创新研究院 | Composite robot suitable for multidimensional operation |
| CN116833724A (en) * | 2023-08-07 | 2023-10-03 | 中国铁建电气化局集团北方工程有限公司 | H-shaped steel column assembly visual guiding method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110936369B (en) | 2022-10-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110936369B (en) | Large-scale workpiece pose accurate measurement and grabbing method based on binocular vision and mechanical arm | |
| CN114434059B (en) | Automatic welding system and method for large structural part with combined robot and three-dimensional vision | |
| CN107571246B (en) | Part assembling system and method based on double-arm robot | |
| CN108972557B (en) | Micro-part pose automatic alignment device and method | |
| CN104690551B (en) | A kind of robot automation's assembly system | |
| CN110919638B (en) | A 3+4 configuration dual-arm collaborative robot processing system and method | |
| CN114728414A (en) | Mobile device with robot, system and machine tool | |
| CN113021017B (en) | Shape-following self-adaptive 3D detection and processing system | |
| CN111409067B (en) | Automatic calibration system and calibration method for robot user coordinate system | |
| CN104044131A (en) | Robot System, Calibration Method, And Method For Producing To-be-processed Material | |
| CN112720458B (en) | System and method for online real-time correction of robot tool coordinate system | |
| CN104044152A (en) | Robot system, method for controlling robot, and method for producing to-be-processed material | |
| CN113500593B (en) | Method for grabbing designated part of shaft workpiece for feeding | |
| Shen et al. | High-precision automated 3-D assembly with attitude adjustment performed by LMTI and vision-based control | |
| CN111687885A (en) | Intelligent double-arm robot system for unordered part assembly and visual guidance method | |
| CN107953333B (en) | Control method and system for calibrating tool at tail end of manipulator | |
| CN110076780B (en) | Robot assembly method and system based on vision and force feedback pose adjustment | |
| CN113182797B (en) | Micro-assembly system based on double macro-micro combined robots | |
| CN114800613B (en) | Precision compensation system and method for online detection of assembly errors of industrial robot | |
| CN216505125U (en) | Visual guidance high-precision positioning system | |
| CN107443404A (en) | End effector, robot and robot controller | |
| CN114006239A (en) | A control method and system for connecting printed circuit board and FPC flexible cable | |
| CN111872921B (en) | Robot double-gripper calibration system and teaching method | |
| CN112571405B (en) | Manipulator and manipulator point position determination method | |
| CN205021615U (en) | Work piece measurement system of robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |