CN111331604A - Machine vision-based valve screwing flexible operation method - Google Patents
Machine vision-based valve screwing flexible operation method Download PDFInfo
- Publication number
- CN111331604A CN111331604A CN202010209058.2A CN202010209058A CN111331604A CN 111331604 A CN111331604 A CN 111331604A CN 202010209058 A CN202010209058 A CN 202010209058A CN 111331604 A CN111331604 A CN 111331604A
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- valve
- screwing
- contact force
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1687—Assembly, peg and hole, palletising, straight line, weaving pattern movement
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a machine vision-based valve screwing flexible operation method. Firstly, constructing a valve hand wheel template image, and marking a valve screwing working point in the template; acquiring image information through a camera, searching SIFT feature points in a visual field image and a template image of the mechanical arm, and matching the feature points; and estimating a conversion matrix between the coordinate systems of the two images by using the matched characteristic point pairs, and further obtaining the screwing operation point of the valve in the visual field image of the mechanical arm. And simultaneously calibrating the pose parameters of the camera, and converting the on-graph coordinates of the screwing working point into the operation space of the mechanical arm to obtain the initial target pose of the mechanical arm. In the valve screwing process, a valve hand wheel generates axial displacement to bring larger axial contact force, and in order to solve the problem, the invention adopts a valve hand wheel position tracking algorithm based on impedance control, and takes the contact force error of the tail end of the mechanical arm and the expected contact force error as feedback quantity to perform online compensation on the target pose of the mechanical arm, thereby improving the force tracking effect.
Description
Technical Field
The invention relates to the technical field of robot motion control, in particular to a target recognition and mechanical arm compliance control method of machine vision.
Background
With the development of the robot technology, the robot intelligent operation technology is applied to various industries. The target object is identified as an important precondition for the intelligent operation of the robot. The target object identification is to accurately position the working point of the mechanical arm, and in the valve screwing task, the working point of the mechanical arm is the position where the tail end screwing hand claw is contacted with the hand wheel of the valve. In the practical application process, in order to position the working point, the valve position and the influence of environmental background information must be recognized in the visual field of the mechanical arm, and a plurality of obstacles exist around the mechanical arm to influence the recognition of the valve hand wheel. How to identify the valve can establish a relation between an image coordinate system and a mechanical arm coordinate system, and an upper screwing working point is converted into a lower coordinate of a base coordinate system, so that the method is a key step of the automatic screwing operation of the mechanical arm.
In the process of screwing the valve, the valve has axial displacement, and if the tail end of the mechanical arm cannot follow the position of the valve, great axial contact force is inevitably brought to the mechanical arm, so that success or failure of screwing tasks is influenced. Therefore, in the process of screwing the valve, the mechanical arm is required to have certain flexibility, and when the axial position of the valve is changed, the tail end of the mechanical arm and the valve hand wheel can make certain yielding to the movement of the valve under the condition of keeping close contact with each other and move along with the valve hand wheel.
Disclosure of Invention
Aiming at the requirements and the defects of the prior art development, the invention provides a valve screwing flexible operation method based on machine vision.
Firstly, the invention protects a valve screwing compliance operation method based on machine vision, and the technical scheme adopted for solving the technical problems is as follows:
1) SIFT feature points are respectively extracted from the image in the mechanical arm view and the valve template image, and feature point pairs between the images are matched according to cosine distances;
2) calculating a coordinate conversion relation between the target image and the template image according to the matched characteristic point pairs, and converting the screwing working point in the template image into the target image;
wherein s isx,syRepresenting a scaling factor, theta representing a rotation angle between the coordinate systems, px,pyRepresenting a translation transformation, x, between coordinate systemsd,ydRepresenting coordinates of characteristic points, x, of the target imagem,ymRepresenting the coordinates of the feature points in the template image.
3) Calibrating a camera, calculating a conversion matrix between a camera coordinate system and a mechanical arm base coordinate system, converting an upper coordinate of a graph into a lower coordinate of the base coordinate system, positioning coordinates of a screwing working point under the base coordinate system, and further calculating an initial target position of the mechanical arm;
4) setting expected contact force in the valve screwing process, and tracking the position of a valve hand wheel; when the contact force of the tail end is not equal to the expected contact force, the relation between the force error and the mechanical arm dynamics is established according to the impedance controller, the pose of the mechanical arm is continuously adjusted, and the error between the contact force of the tail end and the expected contact force is guaranteed to be reduced.
The invention has the beneficial effects that: compared with the prior art, the valve hand wheel screwing device disclosed by the invention has the advantages that the position of the valve hand wheel screwing working point is obtained in the image information through developing research on a machine vision technology, a target pose is provided for the motion control of the mechanical arm, and the autonomous operation of valve screwing is further realized. In order to solve the problem of axial force in the screwing process, the invention utilizes the impedance controller to realize the position tracking of the tail end of the mechanical arm on the valve hand wheel, and in the screwing process, the mechanical arm has axial active flexibility and releases the axial contact force of the tail end.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of the main algorithm;
fig. 2 is a flowchart of the SIFT feature extraction algorithm.
The specific implementation mode is as follows:
the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a valve screwing operation method based on machine vision, including:
vision module
Step 1: making a template image, extracting all key points in the template image, and describing the key points by adopting 128-dimensional vectors;
step 2: extracting all key points and descriptors thereof in the image to be detected, and calculating cosine distances between the template and all key point descriptors in the image to be detected as shown in FIG. 2;
and step 3: based on the symmetry constraint of the valve images and based on two principles of nearest distance and consistency of matching directions, a matching relation is established between key points of the 5 images, and the position of the valve is positioned in the target image according to a matching result.
And 4, step 4: the screwing working point can be directly marked in the template image, and the target image and the template image are subjected to scaling, rotation and translation transformation to obtain the conversion relation between the template image coordinate system and the target image coordinate system, so that the position of the screwing working point in the target image is positioned. The coordinates of the matching point pairs are homogenized, a coordinate transformation matrix T between a template image coordinate system and an image coordinate system to be detected is calculated according to the matching point pairs, and a transformation relation between the two coordinate systems is established:
wherein s isx,syRepresenting a scaling factor, theta representing a rotation angle between the coordinate systems, px,pyRepresenting a translation transformation, x, between coordinate systemsd,ydRepresenting coordinates of characteristic points, x, of the target imagem,ymRepresenting the coordinates of the feature points in the template image. And substituting the matched characteristic point pairs into the formula to perform least square estimation on the coordinate transformation matrix.
And 5: and calibrating the camera, acquiring images of a plurality of calibration plates, calculating an external reference matrix of the camera and a position matrix of the tail end of the corresponding mechanical arm, namely obtaining a conversion matrix between a coordinate system and a camera coordinate system, and further combining image coordinates of the screwing working point to obtain a target position of the mechanical arm for screwing operation under a base coordinate system.
Mechanical arm motion control module
Step 1: the mechanical arm is subjected to motion control, the pose of the screwing operation target obtained by the vision module is reached, and whether the mechanical arm is in contact with the valve or not is judged according to the contact force of the tail end of the mechanical arm;
step 2: and reading the contact force of the tail end through the force sensor, and carrying out mean value filtering to reduce the influence of noise. Inputting the contact force of the tail end and the expected contact force into an impedance controller to control the motion of the mechanical arm;
And step 3: and accumulating the speed and the position adjustment quantity of the tail end of the mechanical arm obtained by the impedance controller into the initial target pose of the mechanical arm, so that the motion control of the mechanical arm by the force error is realized, and the mechanical arm can follow the position of the valve in the screwing process.
Claims (3)
1. A valve screwing compliant operation method based on machine vision is characterized by comprising the following steps:
1) SIFT feature points are respectively extracted from the image in the mechanical arm view and the valve template image, and feature point pairs between the images are matched according to cosine distances;
2) calculating a coordinate conversion relation between the target image and the template image according to the matched characteristic point pairs, and converting the screwing working point in the template image into the target image;
3) calibrating a camera, calculating a conversion matrix between a camera coordinate system and a mechanical arm base coordinate system, converting an upper coordinate of a graph into a lower coordinate of the base coordinate system, positioning coordinates of a screwing working point under the base coordinate system, and further calculating an initial target pose of the mechanical arm;
4) setting expected contact force in the valve screwing process, and tracking the position of a valve hand wheel; when the contact force of the tail end is not equal to the expected contact force, the relation between the force error and the mechanical arm dynamics is established according to the impedance controller, the pose of the mechanical arm is continuously adjusted, and the error between the contact force of the tail end and the expected contact force is guaranteed to be reduced.
2. The machine vision-based valve screwing compliance operation method of claim 1, wherein: the characteristic points of the template image and the target image are extracted and matched, and the conversion relation between the coordinate system of the target image and the coordinate system of the template image is calculated according to the matched characteristic points.
3. The machine vision-based valve screwing compliance operation method of claim 1, wherein: in order to solve the problem of axial force in the screwing process, the invention utilizes the impedance controller to realize the position tracking of the tail end of the mechanical arm on the valve hand wheel, and in the screwing process, the mechanical arm has axial active flexibility and releases the axial contact force of the tail end. And accumulating the speed and the position adjustment quantity of the tail end of the mechanical arm obtained by the impedance controller into the target pose of the mechanical arm, so that the motion control of the mechanical arm by the force error is realized, and the mechanical arm can follow the position of the valve in the screwing process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010209058.2A CN111331604A (en) | 2020-03-23 | 2020-03-23 | Machine vision-based valve screwing flexible operation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010209058.2A CN111331604A (en) | 2020-03-23 | 2020-03-23 | Machine vision-based valve screwing flexible operation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111331604A true CN111331604A (en) | 2020-06-26 |
Family
ID=71176574
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010209058.2A Pending CN111331604A (en) | 2020-03-23 | 2020-03-23 | Machine vision-based valve screwing flexible operation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111331604A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113400304A (en) * | 2021-06-02 | 2021-09-17 | 清华大学 | Acting force-displacement-vision hybrid control method for robot tracheal intubation |
| CN114347035A (en) * | 2022-01-28 | 2022-04-15 | 山东大学 | Robot valve screwing method and system based on teaching learning and flexible control |
| CN114441807A (en) * | 2021-07-22 | 2022-05-06 | 荣耀终端有限公司 | Wiring method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105729468A (en) * | 2016-01-27 | 2016-07-06 | 浙江大学 | Enhanced robot workbench based on multiple depth cameras |
| US20170132807A1 (en) * | 2015-08-06 | 2017-05-11 | Cognex Corporation | System and method for tying together machine vision coordinate spaces in a guided assembly environment |
| CN108000477A (en) * | 2017-12-05 | 2018-05-08 | 哈尔滨工业大学 | A kind of complete main passive compliance robot of pose and screw valve method using the robot |
| CN108161940A (en) * | 2018-02-26 | 2018-06-15 | 哈尔滨工业大学 | Realize that screw operating method is unscrewed in man-machine coordination using space manipulator operating system |
| US20180361589A1 (en) * | 2017-06-16 | 2018-12-20 | Robotiq Inc. | Robotic arm camera system and method |
| CN109940605A (en) * | 2019-01-29 | 2019-06-28 | 中国工程物理研究院激光聚变研究中心 | Merge the submissive assembly system and method for 3D vision and contact force analysis |
-
2020
- 2020-03-23 CN CN202010209058.2A patent/CN111331604A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170132807A1 (en) * | 2015-08-06 | 2017-05-11 | Cognex Corporation | System and method for tying together machine vision coordinate spaces in a guided assembly environment |
| CN105729468A (en) * | 2016-01-27 | 2016-07-06 | 浙江大学 | Enhanced robot workbench based on multiple depth cameras |
| US20180361589A1 (en) * | 2017-06-16 | 2018-12-20 | Robotiq Inc. | Robotic arm camera system and method |
| CN108000477A (en) * | 2017-12-05 | 2018-05-08 | 哈尔滨工业大学 | A kind of complete main passive compliance robot of pose and screw valve method using the robot |
| CN108161940A (en) * | 2018-02-26 | 2018-06-15 | 哈尔滨工业大学 | Realize that screw operating method is unscrewed in man-machine coordination using space manipulator operating system |
| CN109940605A (en) * | 2019-01-29 | 2019-06-28 | 中国工程物理研究院激光聚变研究中心 | Merge the submissive assembly system and method for 3D vision and contact force analysis |
Non-Patent Citations (1)
| Title |
|---|
| 孔韦韦等: "《图像融合技术 基于多分辨率非下采样理论与方法》", 31 July 2015, 西安电子科技大学出版社 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113400304A (en) * | 2021-06-02 | 2021-09-17 | 清华大学 | Acting force-displacement-vision hybrid control method for robot tracheal intubation |
| CN114441807A (en) * | 2021-07-22 | 2022-05-06 | 荣耀终端有限公司 | Wiring method and system |
| CN114347035A (en) * | 2022-01-28 | 2022-04-15 | 山东大学 | Robot valve screwing method and system based on teaching learning and flexible control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110116407B (en) | Flexible robot position and posture measuring method and device | |
| CN111331604A (en) | Machine vision-based valve screwing flexible operation method | |
| Fomena et al. | Distance-based and orientation-based visual servoing from three points | |
| CN103895023A (en) | Mechanical arm tail end tracking and measuring system and method based on coding azimuth device | |
| CN110936355B (en) | Mechanical arm guiding system and method based on visual speed measurement positioning | |
| CN109465829B (en) | Industrial robot geometric parameter identification method based on transformation matrix error model | |
| CN113284179A (en) | Robot multi-object sorting method based on deep learning | |
| CN113910218B (en) | Robot calibration method and device based on kinematic and deep neural network fusion | |
| CN107563432A (en) | A kind of robot multiple-objective recognition methods of view-based access control model shape | |
| CN110463376A (en) | A kind of plug in machine method and slotting machine equipment | |
| CN112947458A (en) | Robot accurate grabbing method based on multi-mode information and computer readable medium | |
| CN115294198A (en) | Vision-based global performance measurement system and method for mechanical arm | |
| Guo et al. | A precision pose measurement technique based on multi-cooperative logo | |
| CN116977434A (en) | Target behavior tracking method and system based on tracking camera | |
| Stemmer et al. | Robust assembly of complex shaped planar parts using vision and force | |
| CN112720449A (en) | Robot positioning device and control system thereof | |
| CN112013868B (en) | Adaptive parameter police dog attitude estimation method based on visual inertial navigation odometer | |
| CN113034418B (en) | Circuit board identification and bonding pad/chip rapid positioning method for electronic industry | |
| Zhang et al. | High-precision pose estimation method of the 3C parts by combining 2D and 3D vision for robotic grasping in assembly applications | |
| Neuberger et al. | Self-initialized Visual Servoing for Accurate End-effector Positioning | |
| Wang et al. | Robotic assembly system guided by multiple vision and laser sensors for large scale components | |
| CN117428777A (en) | Hand-eye calibration method of bag-removing robot | |
| TWI788253B (en) | Adaptive mobile manipulation apparatus and method | |
| CN109146957A (en) | A kind of robot vision control method based on triangle character | |
| EP4113053A1 (en) | Apparatus for correcting assembly deviation and process error thereof using sensing mark, and method of correcting assembly deviation and process error using same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200626 |
|
| WD01 | Invention patent application deemed withdrawn after publication |