CN111482961A - Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator - Google Patents
Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator Download PDFInfo
- Publication number
- CN111482961A CN111482961A CN202010246877.4A CN202010246877A CN111482961A CN 111482961 A CN111482961 A CN 111482961A CN 202010246877 A CN202010246877 A CN 202010246877A CN 111482961 A CN111482961 A CN 111482961A
- Authority
- CN
- China
- Prior art keywords
- manipulator
- coordinate
- coordinates
- target
- movement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000004590 computer program Methods 0.000 claims abstract description 6
- 238000006073 displacement reaction Methods 0.000 claims description 17
- 230000001276 controlling effect Effects 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000002596 correlated effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000012636 effector Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention relates to the technical field of manipulator control, in particular to a positioning control method for manipulator movement, a computer readable storage medium and a device with a manipulator, wherein the computer readable storage medium is stored in the device with the manipulator, the device with the manipulator is provided with a processor, and the method for realizing the positioning control method for manipulator movement when the processor executes a computer program in the computer readable storage medium comprises the following steps: a coordinate acquisition step of acquiring coordinates of the target and pose coordinates of the manipulator by shooting; controlling the manipulator to move from the pose coordinate to the coordinate of the target; the coordinate acquiring step is automatically performed in the traveling step. After the manipulator moves for a certain distance, the actual error of the coordinates between the manipulator and the target can be reduced by acquiring the coordinates of the target again in the pixel coordinate system, the position relation between the coordinates of the target and the pose coordinates of the manipulator is more accurate, and the manipulator can move to the position of the target more accurately.
Description
Technical Field
The invention relates to the technical field of manipulator control, in particular to a positioning control method for manipulator movement, a computer readable storage medium and a device with a manipulator.
Background
The manipulator grabbing operation is one of the main functions of most manipulators, an early manipulator control system can only control the manipulator to move to a preset target position for grabbing through a preset fixed path, and along with the development of a machine vision technology, the manipulator control system can realize real-time grabbing of targets at different positions and different types.
The manipulator control system mainly comprises a vision system and a displacement control system. The vision system carries out camera calibration in advance so as to obtain a pixel coordinate system of the camera, and the vision system carries out hand-eye calibration in advance so as to obtain a conversion relation X between the pixel coordinate system of the camera and a base coordinate system of the manipulator. Camera calibration and hand-eye calibration, as well as a pixel coordinate system of the camera, a base coordinate system of the manipulator and coordinate conversion between the two are all common technical means in machine vision. The manipulator control system acquires an image of an object to be grabbed and the position of the object to be grabbed through a camera of the vision system, performs characteristic processing on the acquired image to obtain required characteristic information, obtains a pixel coordinate of the object to be grabbed in a pixel coordinate system of the camera through the characteristic information, converts the pixel coordinate of the object to be grabbed into a coordinate in a base coordinate system of the manipulator through a conversion relation X, controls the manipulator to move from an initial coordinate to the coordinate of the object to be grabbed, which is obtained by the vision system, in the base coordinate system of the manipulator, and controls an end effector of the manipulator to grab after the manipulator is confirmed to move in place.
The vision system can be divided into an Eye-in-Hand type and an Eye-to-Hand type according to the positional relationship between the camera and the manipulator.
The Eye-in-Hand type camera is installed on a manipulator, and the Hand-Eye calibration algorithm of the installation method is complex and is inconvenient to use on the manipulator with low calculation power. The Eye-to-Hand type is that the camera is separated from the manipulator, the camera is installed at a position outside the manipulator, the position of the camera is not changed along with the movement of the manipulator, and the influence of calibration errors of the camera cannot be directly reduced along with the movement of the manipulator.
Disclosure of Invention
The purpose of the invention is: a method for controlling a manipulator to accurately move to a target position, a computer-readable storage medium storing a program for implementing the method when executed by a processor, and a manipulator apparatus using the medium are provided, which are capable of controlling the manipulator to accurately move to the target position.
The inventor has realized that the error of the pixel coordinates in the pixel coordinate system is positively correlated to the image-taking distance of the camera, and the closer the manipulator is to the target, the smaller the difference between the image-taking distance of the camera to the manipulator and to the target, which can be used to reduce the error difference between the coordinates of the manipulator and the coordinates of the target in the pixel coordinate system.
The positioning control method for movement of the manipulator comprises the following steps:
a coordinate acquisition step of acquiring coordinates of the target and pose coordinates of the manipulator by shooting;
a moving step of controlling the manipulator to move from the pose coordinate to the coordinate of the target;
the coordinate acquiring step is automatically performed in the traveling step.
There is also provided a computer readable storage medium storing a computer program which, when executed by a processor, is capable of implementing the above-described method for controlling the positioning of the movement of a robot arm.
There is also provided an apparatus with a manipulator including a manipulator, a camera for photographing the manipulator and a target, the camera not moving following the manipulator, and a processor controlling the camera to photograph and acquire coordinates of the manipulator and coordinates of the target from an image photographed by the camera, and a computer-readable storage medium as described above, which stores a computer program executable by the processor.
Has the advantages that: in the equipment with the manipulator applying the Eye-to-Hand type vision system, the manipulator movement positioning control method is used, after the manipulator moves a distance from the position and posture coordinate to the coordinate of the target, the distance between the manipulator and the target is reduced in the pixel coordinate system, the coordinate of the target is obtained again at the moment, the error between the coordinate and the current position and posture coordinate of the manipulator is smaller than that before, the actual error of the coordinate between the manipulator and the target is reduced, the position relation between the coordinate of the target and the position and posture coordinate of the manipulator is more accurate, and the manipulator can move to the position of the target more accurately.
Detailed Description
The intelligent kitchen, the intelligent home and the intelligent production environment are all provided with grabbing equipment for taking and placing target objects by using a four-axis mechanical arm and a camera, a clamping jaw is installed at the tail end of the four-axis mechanical arm to take and place the objects, a displacement control system of the four-axis mechanical arm is used for identifying coordinates of the clamping jaw and each joint of the mechanical arm in a base coordinate system and controlling the clamping jaw and each joint to move, a camera is fixed on the grabbing equipment and does not move along with the mechanical arm, and the camera is connected with an Eye-to-Hand type visual system of the grabbing equipment.
In the following, detailed description is given of the process of moving the robot to the target position in the process of grabbing the target object by the four-axis robot.
The controller of the grabbing device starts the four-axis manipulator, controls the camera to shoot the manipulator and the target object to be grabbed, obtains a plane image of the manipulator and the target object, and the grabbing device processes the plane image through the vision system (the method for obtaining the coordinates of the target through image processing in machine vision is a conventional technology of manipulator control, and details are not described here), so that the coordinates of the target object in a pixel coordinate system are obtained, and the initial pose coordinates of the manipulator in the pixel coordinate system before the manipulator starts to move are obtained. The displacement control system plans a movement path for the manipulator to move to the coordinates of the target object (the manipulator path planning is a conventional technology for manipulator control, and is not described herein in detail), and controls the manipulator to move from the initial pose coordinates to the coordinates of the target object. And the displacement control system of the manipulator acquires the displacement coordinates of the manipulator in the base coordinate system of the manipulator in real time.
And the displacement control system of the manipulator controls the manipulator to keep moving to the coordinates of the target object, and the controller of the grabbing equipment controls the camera to acquire the plane images of the manipulator and the target object again, so that the current coordinate Ps of the target object in the pixel coordinate system and the current pose coordinate of the manipulator in the pixel coordinate system are obtained. In a short period of time, the camera continuously performs the operation of acquiring the plane images of the manipulator and the target object again for multiple times to obtain multiple coordinates Ps, and a median value or an average value of the multiple coordinates Ps continuously acquired is taken as the coordinates of the target acquired this time to update the coordinates of the target acquired last time. The process of updating the coordinates of the target is automatically carried out, manual intervention is not needed midway, and the whole-process manipulator keeps moving to the coordinates of the target object without stopping.
The error of the pixel coordinate in the pixel coordinate system is positively correlated with the image-taking distance of the camera, and the closer the manipulator is to the target, the smaller the difference between the image-taking distances of the camera to the manipulator and the target is, and the closer the error generated in the process of converting the coordinate of the manipulator and the target in the pixel coordinate system to the base coordinate system through the conversion relation X is. After the manipulator moves from the initial pose coordinate to the coordinate of the target to a preset degree, for example, after the time that the manipulator has moved reaches one third of the total time required for the manipulator to move from the initial pose coordinate to the coordinate of the target, or after the preset degree is reached that the distance that the manipulator has moved reaches one half of the distance that the manipulator moves from the initial pose coordinate to the coordinate of the target, in the pixel coordinate system, the distance between the manipulator and the target is reduced, the coordinate of the target is obtained again at the moment, the error between the coordinate of the target and the current pose coordinate of the manipulator is smaller than that before, the actual error of the coordinate between the manipulator and the target is reduced, the position relation between the coordinate of the target and the pose coordinate of the manipulator is more accurate, and the manipulator moves to the position of the target more accurately.
When the manipulator is closer to the position of the target object, that is, the distance is smaller, and the subsequent compensation is performed on the coordinates of the target object, the closer the generated error is, the more accurate compensation amount M can be obtained. And a vision system of the grabbing equipment converts the pose coordinate through a conversion relation X between a pixel coordinate system of the camera and a base coordinate system of the manipulator to obtain a relative coordinate R of the pose coordinate in the base coordinate system, at the same moment when the vision system acquires the plane image again as described above, a displacement control system acquires a displacement coordinate of the manipulator in the base coordinate system, acquires a difference value between the relative coordinate R and the displacement coordinate as a compensation quantity M, and compensates the current coordinate of the target by using the compensation quantity M so as to update the coordinate of the target. The relative coordinate R and the displacement coordinate are a group, a plurality of groups are continuously obtained within a period of time, and the median value or the average value of the compensation quantity of all the groups is taken as the compensation quantity M of the coordinate updating step, so that the random error is reduced. And (3) checking whether the relative coordinate R is valid by using the currently acquired coordinate Ps: if the difference between the currently acquired coordinate Ps and the initially acquired coordinate of the target is greater than or equal to the preset threshold, it is determined that the planar image acquired by the vision system at this time has a defect, the currently acquired coordinate Ps cannot be used as the coordinate of the target, and the relative coordinate R acquired in the same planar image with the currently acquired coordinate Ps is also invalid. The compensation amount M is directed from the relative coordinates R to the displacement coordinates, and the coordinates of the target are updated by adding the compensation amount M to the coordinates of the target acquired last time. And the displacement control system of the manipulator controls the manipulator to move from the displacement coordinates to the updated coordinates of the target.
And the updating process of the coordinates of the target is executed for multiple times in the moving process of the manipulator until the clamping jaw of the manipulator reaches the position of the target.
Claims (11)
1. The positioning control method for the movement of the manipulator comprises the following steps:
a coordinate acquisition step of acquiring coordinates of the target and pose coordinates of the manipulator by shooting;
a moving step of controlling the manipulator to move from the pose coordinate to the coordinate of the target;
it is characterized in that the coordinate acquisition step is automatically executed in the advancing step.
2. The method of controlling the positioning of the movement of the robot hand according to claim 1, wherein the coordinate acquiring step is performed before the robot hand starts the movement.
3. The robot hand movement positioning control method according to claim 1, wherein the coordinate acquiring step is automatically performed in the traveling step, specifically, the pose coordinates of the robot hand in the pixel coordinate system at the present time are acquired, the relative coordinates R of the pose coordinates at the present time in the base coordinate system of the robot hand are correspondingly acquired, the displacement coordinates of the robot hand in the base coordinate system at the same time are acquired, the difference between the relative coordinates R and the displacement coordinates is acquired as the compensation amount M, and the current coordinates of the target are compensated by the compensation amount M to update the coordinates of the target.
4. The method of controlling positioning of movement of a robot hand according to claim 1, wherein the robot hand is kept moving while the coordinate acquiring step is automatically performed in the advancing step.
5. The method of controlling positioning of movement of a robot hand according to claim 1, wherein the coordinate acquiring step is automatically performed in the advancing step, specifically, after the robot hand moves from the pose coordinates to the coordinates of the object to a predetermined degree.
6. The method of controlling positioning of movement of a robot hand according to claim 5, wherein the degree of the preset reaching is one third of a total time required for the robot hand to have moved to reach the coordinate movement from the current pose coordinate to the target, or the degree of the preset reaching is one half of a distance that the robot hand has moved to reach the coordinate movement from the current pose coordinate to the target.
7. A positioning control method for robot hand movement according to claim 3, wherein the compensation amount M is directed from the relative coordinate R to the displacement coordinate, and the compensation of the current target coordinate by the compensation amount M is performed by: the coordinates of the target are updated by adding the compensation amount M to the coordinates of the target acquired last time.
8. A positioning control method of robot hand movement according to claim 3, wherein the relative coordinates R and the displacement coordinates are one set, a plurality of sets are successively acquired, and the median or average of the compensation amounts of all sets is taken as the compensation amount M in the current coordinate updating step.
9. The method of controlling positioning of movement of a robot hand according to claim 1, wherein the coordinate acquiring step is automatically performed a plurality of times in the advancing step.
10. A computer-readable storage medium storing a computer program, wherein the computer program is capable of implementing the method for controlling the positioning of the movement of a robot hand according to any one of claims 1 to 9 when executed by a processor.
11. An apparatus with a manipulator comprising a manipulator, a camera for photographing the manipulator and an object, the camera not moving along with the manipulator, and a processor for controlling the camera to photograph and acquire coordinates of the manipulator and coordinates of the object from an image photographed by the camera, characterized by further comprising a computer-readable storage medium as claimed in claim 10, the computer-readable storage medium storing a computer program executable by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010246877.4A CN111482961A (en) | 2020-03-31 | 2020-03-31 | Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010246877.4A CN111482961A (en) | 2020-03-31 | 2020-03-31 | Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111482961A true CN111482961A (en) | 2020-08-04 |
Family
ID=71790107
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010246877.4A Pending CN111482961A (en) | 2020-03-31 | 2020-03-31 | Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111482961A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0929673A (en) * | 1995-07-10 | 1997-02-04 | Mitsubishi Heavy Ind Ltd | Manipulator controller |
| US20130184870A1 (en) * | 2012-01-13 | 2013-07-18 | Yasuhiro Ota | Methods and computer-program products for generating grasp patterns for use by a robot |
| CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
| CN110509281A (en) * | 2019-09-16 | 2019-11-29 | 中国计量大学 | The apparatus and method of pose identification and crawl based on binocular vision |
| CN110717943A (en) * | 2019-09-05 | 2020-01-21 | 中北大学 | Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane |
-
2020
- 2020-03-31 CN CN202010246877.4A patent/CN111482961A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0929673A (en) * | 1995-07-10 | 1997-02-04 | Mitsubishi Heavy Ind Ltd | Manipulator controller |
| US20130184870A1 (en) * | 2012-01-13 | 2013-07-18 | Yasuhiro Ota | Methods and computer-program products for generating grasp patterns for use by a robot |
| CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
| CN110717943A (en) * | 2019-09-05 | 2020-01-21 | 中北大学 | Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane |
| CN110509281A (en) * | 2019-09-16 | 2019-11-29 | 中国计量大学 | The apparatus and method of pose identification and crawl based on binocular vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6966582B2 (en) | Systems and methods for automatic hand-eye calibration of vision systems for robot motion | |
| US8326460B2 (en) | Robot system comprising visual sensor | |
| CN107471218B (en) | Binocular vision-based hand-eye coordination method for double-arm robot | |
| JP2018169403A5 (en) | ||
| WO2018043525A1 (en) | Robot system, robot system control device, and robot system control method | |
| CN108827154B (en) | Robot non-teaching grabbing method and device and computer readable storage medium | |
| CN108399639A (en) | Fast automatic crawl based on deep learning and arrangement method | |
| JP2017505240A (en) | Automatic calibration method for robotic systems using visual sensors | |
| CN110450129B (en) | Carrying advancing method applied to carrying robot and carrying robot thereof | |
| JP2015182144A (en) | Robot system and calibration method of robot system | |
| EP3988254A1 (en) | Robot hand-eye calibration method and apparatus, computing device, medium and product | |
| CN110712194B (en) | Object inspection device, object inspection system, and method of adjusting inspection position | |
| US20200061825A1 (en) | Interference avoidance device and robot system | |
| US20220111533A1 (en) | End effector control system and end effector control method | |
| CN111390910A (en) | Manipulator target grabbing and positioning method, computer readable storage medium and manipulator | |
| CN114519738A (en) | Hand-eye calibration error correction method based on ICP algorithm | |
| JP2006224291A (en) | Robot system | |
| Hvilshøj et al. | Calibration techniques for industrial mobile manipulators: Theoretical configurations and best practices | |
| CN114074331A (en) | Disordered grabbing method based on vision and robot | |
| CN112783076B (en) | Method and system for guiding manipulator positioning based on high-precision displacement sensor | |
| CN111482961A (en) | Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator | |
| CN110533727B (en) | Robot self-positioning method based on single industrial camera | |
| CN114187312A (en) | Target object grabbing method, device, system, storage medium and equipment | |
| CN113858214B (en) | Positioning method and control system for robot operation | |
| JPH09323280A (en) | Control method and system of manupulator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |