CN115519533A - Robot eye calibration method and device, robot and storage medium - Google Patents
Robot eye calibration method and device, robot and storage medium Download PDFInfo
- Publication number
- CN115519533A CN115519533A CN202110715867.5A CN202110715867A CN115519533A CN 115519533 A CN115519533 A CN 115519533A CN 202110715867 A CN202110715867 A CN 202110715867A CN 115519533 A CN115519533 A CN 115519533A
- Authority
- CN
- China
- Prior art keywords
- calibrated
- point
- point location
- marker
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/08—Programme-controlled manipulators characterised by modular constructions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J18/00—Arms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/10—Programme-controlled manipulators characterised by positioning means for manipulator elements
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The application relates to a robot hand-eye calibration method, a robot hand-eye calibration device, a robot and a storage medium. The method comprises the following steps: acquiring a preset number of points to be calibrated; calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in the visual coordinate system after being guided to the corresponding point location to be calibrated through the mechanical arm; matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between a mechanical arm coordinate system and a visual coordinate system; converting each first position to a mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point location to be calibrated; and obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimation position, and re-calibrating the corresponding point location to be calibrated when the matching error which does not meet the preset precision requirement exists. The method can improve the reliability and the precision of the calibration result.
Description
Technical Field
The present application relates to the field of robot technologies, and in particular, to a method and an apparatus for calibrating a hand and an eye of a robot, and a storage medium.
Background
In a robot eye system, the process of obtaining the transformation relationship between the coordinate system of the vision sensor and the coordinate system of the mechanical arm is generally called external reference calibration. The calibration process is widely applied to the field of various robots, particularly various scenes needing to guide the mechanical arm to complete an operation task through a vision technology, and the precision and the reliability of the operation process of the mechanical arm are determined to a great extent by a calibration result.
At present, the calibration process of the hand-eye system is generally a 9-point calibration method, that is, the conversion relationship is obtained by respectively obtaining the positions of 9 spatial points under a mechanical arm coordinate system and a visual coordinate system for matching, and the conversion relationship is used as an external reference calibration result. However, the existing calibration method has the problem that the calibration result depends heavily on the given 9 spatial point positions, and if there is an individual calibration point abnormality, such as exceeding the visual field of the visual sensor, recognition error, etc., the reliability and accuracy of the calibration result will be seriously affected.
Disclosure of Invention
In view of the above, it is necessary to provide a robot eye calibration method, apparatus, robot and storage medium capable of improving reliability and accuracy of calibration results.
A robot hand-eye calibration method, the method comprising:
acquiring a preset number of points to be calibrated, wherein the points to be calibrated represent the designated positions of the marker in a mechanical arm coordinate system;
calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents a position of the marker in a visual coordinate system after the marker is guided to the corresponding point location to be calibrated through a mechanical arm;
matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between the mechanical arm coordinate system and the visual coordinate system;
converting each first position to the mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point location to be calibrated;
and obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimated position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching errors of all the point locations to be calibrated accord with the preset precision requirement.
A robotic hand-eye calibration device, the device comprising:
the system comprises an acquisition module, a calibration module and a calibration module, wherein the acquisition module is used for acquiring a preset number of points to be calibrated, and the points to be calibrated represent the designated positions of a marker in a mechanical arm coordinate system;
the calibration module is used for respectively calibrating each point location to be calibrated to obtain a first position corresponding to each point location to be calibrated, and the first position represents the position of the marker under the visual coordinate system after the marker is guided to the corresponding point location to be calibrated through the mechanical arm;
the matching module is used for matching according to each point location to be calibrated and the corresponding first position thereof to obtain the conversion relation between the mechanical arm coordinate system and the visual coordinate system;
the conversion module is used for converting each first position to the mechanical arm coordinate system based on the conversion relation to obtain an estimated position corresponding to each point location to be calibrated;
and the correction module is used for obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimation position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching error of all the point locations to be calibrated accords with the preset precision requirement.
A robot comprising a memory storing a computer program and a processor implementing the following steps when executing the computer program:
acquiring a preset number of points to be calibrated, wherein the points to be calibrated represent the designated positions of the markers in a mechanical arm coordinate system;
calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in a visual coordinate system after the marker is guided to the corresponding point location to be calibrated through a mechanical arm;
matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between the mechanical arm coordinate system and the visual coordinate system;
converting each first position to the mechanical arm coordinate system based on the conversion relation to obtain an estimated position corresponding to each point location to be calibrated;
and obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimated position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching errors of all the point locations to be calibrated accord with the preset precision requirement.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a preset number of points to be calibrated, wherein the points to be calibrated represent the designated positions of the marker in a mechanical arm coordinate system;
calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in a visual coordinate system after the marker is guided to the corresponding point location to be calibrated through a mechanical arm;
matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between the mechanical arm coordinate system and the visual coordinate system;
converting each first position to the mechanical arm coordinate system based on the conversion relation to obtain an estimated position corresponding to each point location to be calibrated;
and obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimated position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching errors of all the point locations to be calibrated accord with the preset precision requirement.
According to the robot hand-eye calibration method, the robot hand-eye calibration device, the robot and the storage medium, a preset number of points to be calibrated are obtained, and the points to be calibrated represent the designated positions of the marker in the mechanical arm coordinate system; calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in the visual coordinate system after being guided to the corresponding point location to be calibrated through the mechanical arm; matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between a mechanical arm coordinate system and a visual coordinate system; converting each first position to a mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point location to be calibrated; and when the matching error which does not accord with the preset precision requirement exists, the corresponding point position to be calibrated is calibrated again until the matching errors of all the point positions to be calibrated accord with the preset precision requirement. Therefore, possible error points or abnormal points in the calibration process can be found out by evaluating the matching errors of the point positions to be calibrated, and then the problems in the calibration process can be corrected by re-calibrating until the matching errors meet the preset precision requirement, so that the reliability and precision of the calibration result are improved.
Drawings
FIG. 1 is a schematic flow chart of a robot hand-eye calibration method in one embodiment;
FIG. 2 is a schematic flow chart illustrating a step of obtaining a preset number of point locations to be calibrated in one embodiment;
FIG. 3 is a schematic diagram illustrating the positions of points to be calibrated in the visual field of the vision sensor according to an embodiment;
FIG. 4 is a schematic flow chart illustrating a step of correcting a point location to be calibrated in one embodiment;
FIG. 5 is a schematic flow chart of a robot eye calibration method in one embodiment;
fig. 6 is a block diagram of a robot hand-eye calibration apparatus according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
The robot hand-eye calibration method can be applied to a robot, the robot is provided with the vision sensor, the vision sensor is rigidly connected with a mechanical arm base of the robot in the calibration process, namely the relative position is unchanged, and the aim of robot hand-eye calibration is to obtain the position conversion relation between the vision sensor and the mechanical arm base.
In one embodiment, as shown in fig. 1, a robot hand-eye calibration method is provided, which includes the following steps S102 to S110.
S102, acquiring a preset number of points to be calibrated, wherein the points to be calibrated represent the designated positions of the marker in the mechanical arm coordinate system.
The point position to be calibrated refers to a position of a spatial point to be calibrated in the calibration process, and can also be understood as a position of a spatial point through which a marker needs to pass and be photographed. Specifically, the marker may be mounted at the end of a robot arm to be calibrated, the points to be calibrated are sequentially sent to the robot arm, the marker is guided by the robot arm to sequentially pass through all the points to be calibrated, when the marker reaches the point to be calibrated, the marker is considered to reach an appointed position, that is, the position of the marker under the coordinate system of the robot arm at this time is considered to be the point to be calibrated.
The number and the arrangement mode of the point positions to be calibrated can be set according to actual requirements, and the point positions are not limited, so long as the number of the subsequently obtained point pairs is enough to solve the conversion relation between the mechanical arm coordinate system and the visual coordinate system. The calibration accuracy is improved more easily by more point pairs, and the interference of individual error points on the final calibration result can be reduced, but more calibration time is consumed. In one embodiment, the predetermined number is 9, which is beneficial to balance calibration accuracy and calibration time cost.
And S104, calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in the visual coordinate system after the marker is guided to the corresponding point location to be calibrated through the mechanical arm.
In the calibration process, when the mechanical arm guides the marker to move to a point to be calibrated, the visual sensor starts to acquire an image, specifically, the marker entering the visual field is photographed, and the position of the marker, namely the position of the marker in the visual coordinate system, is obtained from the image, namely the first position corresponding to the point to be calibrated. Recording the corresponding relation between each point location to be calibrated and the corresponding first position thereof to form a plurality of groups of point pairs, wherein each group of point pairs comprises the point location to be calibrated and the first position corresponding to the point location to be calibrated.
And S106, matching according to each point location to be calibrated and the corresponding first position of the point location to be calibrated to obtain a conversion relation between a mechanical arm coordinate system and a visual coordinate system.
The transformation relation can be a transformation matrix of the mechanical arm coordinate system relative to the visual coordinate system, and after a plurality of point pairs are obtained through the calibration process, the relative attitude relation between the visual coordinate system and the mechanical arm coordinate system can be solved based on the plurality of point pairs to obtain the transformation matrix. The method for solving the transformation matrix may be performed in any manner that is currently available and may appear later, and is not limited herein.
And S108, converting each first position to a mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point to be calibrated.
After obtaining the transformation relationship between the robot coordinate system and the visual coordinate system, the inverse calculation may be performed based on the transformation relationship and each first position, so as to transform each first position into the robot coordinate system, and obtain a corresponding position (expressed by an estimated position so as to be distinguished from the specified position in the foregoing) of each first position in the robot coordinate system. It can be understood that, since each first position corresponds to one point to be calibrated, for any first position, the point to be calibrated corresponding to the estimated position calculated based on the first position is the point to be calibrated corresponding to the first position.
Specifically, the inverse calculation formula may be as follows: p ig ＝MC i Wherein, C i Representing a first position corresponding to the ith point to be calibrated, M representing a conversion matrix, P ig And representing the estimation position corresponding to the ith point to be calibrated.
S110, obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimation position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching errors of all the point locations to be calibrated accord with the preset precision requirement.
The conversion relation is obtained by solving based on all the points to be calibrated and the corresponding first positions thereof, the overall matching condition is reflected, and the calibration process of each point to be calibrated is relatively independent, so that the estimated position obtained by reverse calculation based on the conversion relation and each first position and the corresponding point to be calibrated may have a certain deviation, and the deviation degree may be different.
The matching error of each point location to be calibrated can be determined according to the deviation between each point location to be calibrated and the corresponding estimated position. Specifically, the matching error calculation formula may be as follows: e.g. of the type i ＝P ig –P i Wherein e is i Represents the matching error of the ith point position to be calibrated, P ig Represents the estimated position, P, corresponding to the ith point location to be calibrated i And representing the ith point to be calibrated.
The matching error of the point location to be calibrated is used for representing whether the calibration process of the point location to be calibrated has problems, and the preset precision requirement is used for evaluating the matching error. When the matching error does not meet the preset precision requirement, the calibration process of the corresponding point location to be calibrated is considered to have problems (for example, the visual algorithm is mistakenly identified, the marker shakes due to the movement of the mechanical arm, the mechanical arm starts to identify when the mechanical arm is not in position, and the like), if the problems exist, the point location to be calibrated is considered to be an error point or an abnormal point, and needs to be calibrated again, so that the adjustment of the local calibration content is realized. And after the error points or the abnormal points are calibrated again, the conversion relation is changed, and then the matching errors of all the point positions to be calibrated are calculated again until the matching errors of all the point positions to be calibrated meet the preset precision requirement. And when the matching errors of all the points to be calibrated meet the preset precision requirement, outputting the conversion matrix at the moment as a final external reference calibration result.
In the robot eye calibration method, a preset number of points to be calibrated are obtained, and the points to be calibrated represent the designated positions of the markers in the mechanical arm coordinate system; calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in the visual coordinate system after being guided to the corresponding point location to be calibrated through the mechanical arm; matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between a mechanical arm coordinate system and a visual coordinate system; converting each first position to a mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point location to be calibrated; and obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimated position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching errors of all the point locations to be calibrated accord with the preset precision requirement. Therefore, possible error points or abnormal points in the calibration process can be found out by evaluating the matching error of the point location to be calibrated, and then the problems in the calibration process can be corrected by re-calibrating until the matching error meets the preset precision requirement, so that the reliability and precision of the calibration result are improved.
In one embodiment, when the matching error of each point location to be calibrated is greater than a threshold value, judging that the matching error which does not meet the preset precision requirement exists; when the matching error of each point location to be calibrated is less than or equal to a threshold value, judging that the matching errors of all the point locations to be calibrated meet the preset precision requirement; the step of recalibrating the corresponding point location to be calibrated may specifically be: and selecting the maximum matching error from the matching errors larger than the threshold value, and re-calibrating the point position to be calibrated corresponding to the maximum matching error.
The point location to be calibrated and the corresponding estimated location thereof are both locations under the coordinates of the mechanical arm, and can be represented by a three-dimensional coordinate vector, and accordingly, the matching error can also be represented as a three-dimensional coordinate vector, where the matching error is greater than a threshold, which may mean that the coordinate of any one or more dimensions in the three-dimensional coordinate vector is greater than the threshold, or that the modulus of the three-dimensional coordinate vector is greater than the threshold, and this is not limited here. The matching error is less than or equal to the threshold, which may mean that the coordinates of all dimensions in the three-dimensional coordinate vector are less than or equal to the threshold, or a modulus of the three-dimensional coordinate vector is less than or equal to the threshold, which is not limited herein. In addition, the size of the threshold may be set according to actual requirements, and is not limited herein. It should be noted that a plurality of matching errors larger than the threshold may exist in the matching error of each point location to be calibrated, the largest matching error among the plurality of matching errors larger than the threshold is found, and the point location to be calibrated corresponding to the largest matching error is recalibrated.
In this embodiment, considering that the conversion relationship is obtained by solving based on all the points to be calibrated and the corresponding first positions thereof, there may be a situation that only one calibration process of a certain point to be calibrated is abnormal but the matching errors of a plurality of points to be calibrated do not meet the preset precision requirement at the same time, so that only the point to be calibrated corresponding to the maximum matching error is recalibrated during each iteration, which can avoid that the point to be calibrated which is not abnormal in the original calibration process is recalibrated to cause a larger matching error, thereby improving the correction efficiency and correction precision of the calibration result.
In an embodiment, as shown in fig. 2, the step of obtaining the preset number of point locations to be calibrated may specifically include the following steps S202 to S204.
S202, acquiring appointed central point positions and appointed relative positions of a preset number, wherein the appointed central point positions meet the following conditions: when the marker is moved to a designated center point position, the marker is in the center region of the visual sensor field of view.
When the mechanical arm guides the marker to move to different positions, the marker appears in different situations in the visual sensor visual field, and may not be in the visual sensor visual field, or may be in the visual sensor visual field, and when the marker is in the visual sensor visual field, the marker may be in the central area of the visual sensor visual field, or may be in the edge area of the visual sensor visual field. Based on this, when the marker is observed to be in the central area of the visual sensor field of view, the position of the marker in the mechanical arm coordinate system at this time is recorded as the designated center point position. The designated relative positions refer to positions relative to the designated central points, and the number of the designated relative positions is the same as the number of the points to be calibrated which need to be calibrated.
And S204, adding the specified central point positions to the specified relative positions respectively to obtain the preset number of points to be calibrated.
The designated central point position and each designated relative position can be represented by a three-dimensional coordinate vector, the designated central point position and the designated relative position are added, specifically, coordinates of all dimensions are added respectively, the designated relative positions are different, and the obtained point positions to be calibrated are also different. For example, assume that a center point location (P) is specified 0 ) Is represented by (X) 0 ,Y 0 ,Z 0 ) Specifying the relative position (Δ P) i ) Is represented by (Delta X) i ,ΔY i ,ΔZ i ) Then point location (P) is to be calibrated i ＝P 0 +ΔP i ) Is represented by (X) 0 +ΔX i ,Y 0 +ΔY i ,Z 0 +ΔZ i ). As shown in fig. 3, a schematic diagram of the positions of the points to be calibrated in the visual field of the visual sensor is provided, where the schematic diagram includes 9 points to be calibrated, which are respectively P 1 ～P 9 The designated center point is P 5 At a position in the central region of the visual field of the vision sensor, P 5 Dividing P for the center point of the 9 point location combinations to be calibrated 5 Other points to be marked surround P 5 Around, when actually calibrating, only P needs to be guaranteed 5 And the other points to be calibrated are uniformly distributed in the visual field of the visual sensor.
In the embodiment, the point locations to be calibrated are obtained by designating the central point location and the relative position, so that the point locations to be calibrated based on the designated central point location can be uniformly distributed in the visual field range of the visual sensor as much as possible, and simultaneously, the point locations to be calibrated can cover as large a range as possible in the visual field of the visual sensor, thereby being beneficial to improving the reliability and precision of the calibration result and being beneficial to the subsequent adjustment of the point locations to be calibrated.
In an embodiment, the step of calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated may specifically include: for each point to be calibrated, the following calibration operations are carried out: and controlling the mechanical arm to guide the marker to move towards the point location to be calibrated, controlling the visual sensor to collect an image when the marker is detected to move in place, and acquiring the position of the marker in a visual coordinate system based on the image to be used as a first position corresponding to the point location to be calibrated.
By taking 9 points to be calibrated as an example for explanation, a calibration sequence of the 9 points to be calibrated is preset, and the points to be calibrated are sequentially expressed as P 1 ～P 9 Sequentially adding P 1 ～P 9 Sending to a mechanical arm, and guiding the markers to move to P in sequence by the mechanical arm 1 ～P 9 When the marker moves to a point to be calibrated, the visual sensor is controlled to collect an image, and the position of the marker in the visual coordinate system is obtained based on the image, so that P is obtained 1 ～P 9 Respectively corresponding first positions, respectively denoted as C 1 ～C 9 。
In one embodiment, the step of controlling the vision sensor to acquire the image when the marker is detected to move to the position may specifically include the following steps: when the marker is detected to move in place, judging whether the marker appears in the visual field of the visual sensor; and controlling the vision sensor to acquire the image when the marker is present in the visual field of the vision sensor.
When the marker is detected to move in place, the marker may not appear in the visual field of the visual sensor, and calibration may be continued without adjustment, which may cause abnormal calibration and affect the final calibration result. Therefore, when the marker appears in the visual field of the visual sensor, the visual sensor is controlled to acquire the image, and the reliability and the precision of the calibration result are ensured.
In one embodiment, if the marker is not in the visual sensor field of view, the point to be calibrated is corrected, and the mechanical arm is controlled to guide the marker to move towards the corrected point to be calibrated until the marker is in the visual sensor field of view.
When the fact that the marker moves in place is detected, if the marker does not appear in the visual sensor vision field, the possible reason is that the relative position coordinates of the point to be calibrated and the designated central point are too large, so that the marker exceeds the sensor vision field, as shown in fig. 3, P 9 Out of the field of view of the sensor, pair P 9 Correcting, specifically, P can be 9 To the centre point P 5 Close to, P 9 Corrected to obtain P 9 '. The control mechanical arm guides the marker to move towards the corrected point position to be calibrated, and the visual sensor is controlled to collect an image until the marker appears in the visual field of the visual sensor, so that the reliability and the precision of a calibration result are guaranteed.
In an embodiment, as shown in fig. 4, the step of correcting the point to be calibrated may specifically include the following steps S402 to S404.
S402, correcting the appointed relative position corresponding to the point location to be calibrated based on the preset offset proportion, and obtaining the corrected relative position corresponding to the point location to be calibrated.
The preset offset proportion is used for representing the correction amplitude of the specified relative position, and the value range of the preset offset proportion is 0-1 (0 and 1 are not included). It can be understood that the larger the preset offset proportion is, the smaller the correction amplitude of the specified relative position is, and correspondingly, the smaller the correction amplitude of the point location to be calibrated is; the smaller the preset deviation ratio is, the finger-to-fingerThe larger the correction amplitude of the fixed relative position is, the larger the correction amplitude of the point position to be calibrated is correspondingly. Specifically, the calculation formula of the corrected relative position may be as follows: delta P i ′＝μΔP i Wherein, Δ P i Showing the designated relative position corresponding to the ith point position to be calibrated, mu showing the preset deviation proportion, and delta P i ' represents the corrected relative position corresponding to the ith point to be calibrated.
And S404, adding the specified point location and the corrected relative position corresponding to the point location to be calibrated to obtain the corrected point location to be calibrated.
Specifically, the corrected calculation formula of the point location to be calibrated may be as follows: p is i ′＝P 0 +ΔP i ′＝P 0 +μΔP i . It should be noted that the specific value of μmay be set according to actual requirements, and is not limited herein. The larger the value of mu is, the more the number of iterations required for moving the point position beyond the visual field of the visual sensor to enter the visual field is, so that the calibration speed is slowed down; the smaller the value of mu is, the point beyond the visual field of the visual sensor may be too close to the central point, resulting in the reduction of calibration accuracy.
In this embodiment, for a point location to be calibrated that exceeds the visual field range of the visual sensor, the designated relative position corresponding to the point location to be calibrated is reduced, so that the point location to be calibrated approaches the designated central point, and thus the point location to be calibrated can accurately enter the visual field range of the visual sensor, and the point location to be calibrated can be corrected accurately in real time in the calibration process. In addition, the correction method for the point location to be calibrated in the embodiment can be suitable for hardware structure change scenes, for example, when the relative position of the visual sensor and the mechanical arm changes, the point location to be calibrated with problems can be corrected in the calibration process through the embodiment, the situation that a fixed set of calibration point location fails to work due to structural change and cannot be normally used is avoided, the dependence of frequent design on professionals of the calibration point location is avoided, and the maintenance cost is reduced.
In one embodiment, the marker is a two-dimensional code. The marker is used for matching common points on a visual coordinate system and a mechanical arm coordinate system, the traditional marker is generally a pattern with a specific color, a sharp end and the like, but the marker needs a customized visual algorithm to complete a target identification and positioning process, the accuracy and reliability of the identification process are necessarily dependent on the personal ability of an algorithm developer, uncontrollable factors of a calibration process are increased, meanwhile, the marker also has the problems, the marker with fixed color can be interfered by similar colors in illumination and background, and the sharp end can be affected by the defocusing of a visual sensor. Therefore, the two-dimensional code is used as the marker in the embodiment, and the two-dimensional code recognition algorithm is highly standardized, strong in environment adaptability, and high in stability, reliability and precision.
In one embodiment, apriltag two-dimensional codes or Aruco two-dimensional codes are used as markers. Through targeted optimization, the identification reliability of the two-dimensional code is greatly improved compared with that of a common two-dimensional code, and depth information can be provided under the conditions that a depth camera is not relied on and only a single two-dimensional visual sensor exists, so that the spatial position of a marker in a visual coordinate system is directly obtained, the precision and the reliability of a calibration process are improved, and the complexity of code development is reduced.
Specifically, the identification process of the marker by the visual sensor comprises the following two schemes: firstly, for a visual system with a depth camera and a two-dimensional camera, firstly, identifying a two-dimensional code marker through the two-dimensional camera to obtain a coordinate of the marker in a pixel coordinate system, and then, obtaining a spatial position of the marker in the visual coordinate system of the depth camera through a matching algorithm; secondly, for a vision system only provided with a two-dimensional camera, internal parameters of the two-dimensional camera can be obtained in advance through a Zhang-Yongyou calibration method, and then the spatial position of the two-dimensional camera under a coordinate system of the two-dimensional camera can be directly identified based on the characteristics of the two-dimensional code marker.
It should be noted that the aprilat two-dimensional code or the ArUco two-dimensional code used in the calibration process may be replaced by other markers, such as other two-dimensional codes, calibration checkerboard, and other characteristic highly-regular and mature and general special patterns of the recognition algorithm, in the case of a depth camera or other visual sensors with the capability of directly feeding back spatial coordinates.
In one embodiment, as shown in fig. 5, a robot hand-eye calibration method is provided, which includes the following steps S501 to S506.
S501, acquiring appointed central point positions and appointed relative positions of a preset number, wherein the appointed central point positions meet the following conditions: when the marker is moved to a designated center point position, the marker is in a center region of the visual sensor field of view.
And S502, adding the specified central point positions to the specified relative positions respectively to obtain the preset number of points to be calibrated.
S503, for each point to be calibrated, executing the following calibration operation: the method comprises the steps of controlling a mechanical arm to guide a marker to move towards a point to be calibrated, judging whether the marker is in a visual field of a visual sensor when the marker is detected to move in place, correcting the point to be calibrated if the marker is not in the visual field of the visual sensor, controlling the mechanical arm to guide the marker to move towards the corrected point to be calibrated until the marker is in the visual field of the visual sensor, controlling a visual sensor to collect an image, and acquiring the position of the marker in a visual coordinate system based on the image to serve as a first position corresponding to the point to be calibrated.
S504, matching is carried out according to each point position to be calibrated and the corresponding first position of the point position to be calibrated, and the conversion relation between the mechanical arm coordinate system and the visual coordinate system is obtained.
And S505, converting each first position to a mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point location to be calibrated.
S506, obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimation position, when the matching error of each point location to be calibrated is larger than the threshold value, selecting the maximum matching error from the matching errors larger than the threshold value, re-calibrating the point location to be calibrated corresponding to the maximum matching error until the matching errors of all the point locations to be calibrated are smaller than or equal to the threshold value, and outputting the finally obtained conversion relation as the calibration result.
For specific description of steps S501 to S506, reference may be made to the foregoing embodiments, which are not described herein again. In the embodiment, on one hand, possible error points or abnormal points in the calibration process can be found out by evaluating the matching error of the point location to be calibrated, and then problems in the calibration process can be corrected by re-calibrating until the matching error meets the preset precision requirement, so that the reliability and precision of the calibration result are improved; on the other hand, the point location to be calibrated is obtained by designating the central point location and the relative position, so that the point location to be calibrated based on the designated central point location can be uniformly distributed in the visual field range of the visual sensor as much as possible, and simultaneously, the point location to be calibrated can cover the range as large as possible in the visual field of the visual sensor, thereby being beneficial to improving the reliability and the precision of the calibration result and being beneficial to the subsequent adjustment of the point location to be calibrated; on the other hand, the point location to be calibrated which exceeds the visual field range of the visual sensor is corrected in real time in the calibration process, the situation that a fixed set of calibration point location fails to work due to structural change and cannot be normally used is avoided, the dependence of frequent design of the calibration point location on professionals is avoided, and the maintenance cost is reduced.
It should be noted that the above embodiment may be applied to not only the calibration of a three-dimensional space to obtain a spatial transformation relationship between two three-dimensional coordinate systems, but also the calibration of a two-dimensional plane to obtain a perspective transformation relationship between two-dimensional plane coordinate systems in the space. For example, two-dimensional plane calibration is realized by limiting the movement of the mechanical arm in a certain fixed plane, specifically, in a visual coordinate system, a third dimension containing a fixed value can be added, in a mechanical arm coordinate system, a limiting condition can be added to the coordinate of the three dimensions to ensure that the coordinate is always located on a specific plane, and then the perspective transformation relationship between the camera plane and the mechanical arm plane is solved.
It should be understood that, although the steps in the flowcharts related to the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in each flowchart related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
In one embodiment, as shown in fig. 6, there is provided a robot hand-eye calibration apparatus 600, comprising: the system comprises an acquisition module 610, a calibration module 620, a matching module 630, a conversion module 640 and a correction module 650, wherein:
the obtaining module 610 is configured to obtain a preset number of to-be-calibrated point locations, where the to-be-calibrated point locations represent specified positions of the marker in a robot arm coordinate system.
The calibration module 620 is configured to calibrate each point location to be calibrated respectively, to obtain a first position corresponding to each point location to be calibrated, where the first position represents a position of the marker in the visual coordinate system after the marker is guided to the corresponding point location to be calibrated by the mechanical arm.
The matching module 630 is configured to perform matching according to each point location to be calibrated and the corresponding first location thereof, so as to obtain a conversion relationship between the mechanical arm coordinate system and the visual coordinate system.
And the converting module 640 is configured to convert each first position to a mechanical arm coordinate system based on the conversion relationship, and obtain an estimated position corresponding to each point location to be calibrated.
And the correcting module 650 is configured to obtain a matching error of each point location to be calibrated according to a deviation between each point location to be calibrated and the corresponding estimated position, and recalibrate the corresponding point location to be calibrated when there is a matching error that does not meet the preset precision requirement until the matching errors of all the point locations to be calibrated meet the preset precision requirement.
In one embodiment, the apparatus further comprises a decision module for: when the matching error of each point location to be calibrated is larger than the threshold value, judging that the matching error which does not meet the preset precision requirement exists; and when the matching error of each point location to be calibrated is less than or equal to the threshold value, judging that the matching errors of all the point locations to be calibrated meet the preset precision requirement. The correction module 650 is specifically configured to, when recalibrating the corresponding point location to be calibrated: and selecting the maximum matching error from the matching errors larger than the threshold value, and re-calibrating the point to be calibrated corresponding to the maximum matching error.
In an embodiment, when the obtaining module 610 obtains a preset number of point locations to be calibrated, it is specifically configured to: acquiring appointed central point positions and appointed relative positions of a preset number, wherein the appointed central point positions meet the following conditions: when the marker moves to the designated center point, the marker is in the center region of the visual sensor field of view; and adding the specified central point positions and the specified relative positions respectively to obtain the preset number of points to be calibrated.
In an embodiment, when the calibration module 620 performs calibration on each point location to be calibrated to obtain a first position corresponding to each point location to be calibrated, the calibration module is specifically configured to: for each point to be calibrated, the following calibration operations are carried out: and controlling the mechanical arm to guide the marker to move towards the point location to be calibrated, controlling the visual sensor to collect an image when the marker is detected to move in place, and acquiring the position of the marker in a visual coordinate system based on the image to be used as a first position corresponding to the point location to be calibrated.
In one embodiment, the calibration module 620 is specifically configured to, when detecting that the marker is moved into position and controlling the vision sensor to acquire the image: when the marker is detected to move in place, judging whether the marker appears in the visual field of the visual sensor; and controlling the vision sensor to acquire the image when the marker is present in the vision sensor field of view.
In one embodiment, the calibration module 620 is further configured to: and if the marker does not appear in the visual sensor field of view, correcting the point to be calibrated, and controlling the mechanical arm to guide the marker to move towards the corrected point to be calibrated until the marker appears in the visual sensor field of view.
In one embodiment, the calibration module 620 is specifically configured to, when correcting the point to be calibrated: correcting the specified relative position corresponding to the point location to be calibrated based on the preset offset proportion to obtain a corrected relative position corresponding to the point location to be calibrated; and adding the specified center point position and the corrected relative position corresponding to the point position to be calibrated to obtain the corrected point position to be calibrated.
In one embodiment, the marker is a two-dimensional code.
For specific limitations of the robot hand-eye calibration device, reference may be made to the above limitations of the robot hand-eye calibration method, which are not described herein again. The various modules in the robotic eye calibration device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In an embodiment, a robot is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above-described method embodiments when executing the computer program.
In an embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, performs the steps of the various method embodiments described above.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the respective method embodiment as described above.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the steps in the various method embodiments described above.
It should be understood that the terms "first", "second", etc. in the above-described embodiments are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A robot hand-eye calibration method is characterized by comprising the following steps:
acquiring a preset number of points to be calibrated, wherein the points to be calibrated represent the designated positions of the markers in a mechanical arm coordinate system;
calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in a visual coordinate system after the marker is guided to the corresponding point location to be calibrated through a mechanical arm;
matching according to each point location to be calibrated and the corresponding first position thereof to obtain a conversion relation between the mechanical arm coordinate system and the visual coordinate system;
converting each first position to the mechanical arm coordinate system based on the conversion relation, and obtaining an estimated position corresponding to each point location to be calibrated;
and obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimated position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching errors of all the point locations to be calibrated accord with the preset precision requirement.
2. The method according to claim 1, characterized in that when a matching error larger than a threshold exists in the matching errors of the point locations to be calibrated, it is determined that a matching error which does not meet a preset precision requirement exists;
when the matching error of each point location to be calibrated is smaller than or equal to the threshold value, determining that the matching error of all the point locations to be calibrated meets the preset precision requirement;
recalibrating the corresponding points to be calibrated, comprising the following steps:
and selecting the maximum matching error from the matching errors larger than the threshold value, and re-calibrating the point position to be calibrated corresponding to the maximum matching error.
3. The method according to claim 1, wherein obtaining a preset number of points to be calibrated comprises:
acquiring appointed central point positions and appointed relative positions of a preset number, wherein the appointed central point positions meet the following conditions: when the marker is moved to the designated center point, the marker is in a central region of a visual sensor field of view;
and adding the specified central point positions to each specified relative position respectively to obtain the preset number of points to be calibrated.
4. The method according to claim 1, wherein calibrating each point location to be calibrated respectively to obtain a first position corresponding to each point location to be calibrated, comprises:
for each point to be calibrated, performing the following calibration operations: and controlling a mechanical arm to guide the marker to move towards the point location to be calibrated, controlling a visual sensor to collect an image when the marker is detected to move in place, and acquiring the position of the marker in a visual coordinate system based on the image to be used as a first position corresponding to the point location to be calibrated.
5. The method of claim 4, wherein controlling a vision sensor to capture an image when the marker is detected to be moved into position comprises:
when the marker is detected to move in place, judging whether the marker is in the visual field of the visual sensor;
controlling a vision sensor to acquire an image when the marker is present within the vision sensor field of view.
6. The method of claim 5, further comprising:
and if the marker does not appear in the visual sensor visual field, correcting the point position to be calibrated, and controlling the mechanical arm to guide the marker to move towards the corrected point position to be calibrated until the marker appears in the visual sensor visual field.
7. The method according to claim 6, wherein obtaining a preset number of points to be calibrated comprises:
acquiring appointed central point positions and appointed relative positions of a preset number, wherein the appointed central point positions meet the following conditions: when the marker is moved to the designated center point, the marker is in a central region of a visual sensor field of view;
adding the specified central point positions to each specified relative position respectively to obtain the preset number of points to be calibrated;
correcting the point location to be calibrated, including:
correcting the specified relative position corresponding to the point location to be calibrated based on a preset offset proportion to obtain a corrected relative position corresponding to the point location to be calibrated;
and adding the specified center point position and the corrected relative position corresponding to the point position to be calibrated to obtain the corrected point position to be calibrated.
8. The method of any one of claims 1 to 7, wherein the marker is a two-dimensional code.
9. A robot hand-eye calibration device, characterized in that the device comprises:
the system comprises an acquisition module, a calibration module and a calibration module, wherein the acquisition module is used for acquiring a preset number of points to be calibrated, and the points to be calibrated represent the designated positions of a marker in a mechanical arm coordinate system;
the calibration module is used for respectively calibrating each point location to be calibrated to obtain a first position corresponding to each point location to be calibrated, wherein the first position represents the position of the marker in a visual coordinate system after the marker is guided to the corresponding point location to be calibrated through the mechanical arm;
the matching module is used for matching according to each point location to be calibrated and the corresponding first position thereof to obtain the conversion relation between the mechanical arm coordinate system and the visual coordinate system;
the conversion module is used for converting each first position to the mechanical arm coordinate system based on the conversion relation to obtain an estimated position corresponding to each point location to be calibrated;
and the correction module is used for obtaining the matching error of each point location to be calibrated according to the deviation between each point location to be calibrated and the corresponding estimation position, and when the matching error which does not accord with the preset precision requirement exists, re-calibrating the corresponding point location to be calibrated until the matching error of all the point locations to be calibrated accords with the preset precision requirement.
10. A robot comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110715867.5A CN115519533A (en) | 2021-06-24 | 2021-06-24 | Robot eye calibration method and device, robot and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110715867.5A CN115519533A (en) | 2021-06-24 | 2021-06-24 | Robot eye calibration method and device, robot and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115519533A true CN115519533A (en) | 2022-12-27 |
Family
ID=84694058
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110715867.5A Pending CN115519533A (en) | 2021-06-24 | 2021-06-24 | Robot eye calibration method and device, robot and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115519533A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116878386A (en) * | 2023-09-06 | 2023-10-13 | 北京华卓精科科技股份有限公司 | Calibration method and calibration device for up-down alignment visual device |
-
2021
- 2021-06-24 CN CN202110715867.5A patent/CN115519533A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116878386A (en) * | 2023-09-06 | 2023-10-13 | 北京华卓精科科技股份有限公司 | Calibration method and calibration device for up-down alignment visual device |
| CN116878386B (en) * | 2023-09-06 | 2023-12-08 | 北京华卓精科科技股份有限公司 | Calibration method and calibration device for up-down alignment visual device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10112301B2 (en) | Automatic calibration method for robot systems using a vision sensor | |
| US20230381954A1 (en) | Robotic system with enhanced scanning mechanism | |
| US5319443A (en) | Detected position correcting method | |
| US11667036B2 (en) | Workpiece picking device and workpiece picking method | |
| KR20180120647A (en) | System and method for tying together machine vision coordinate spaces in a guided assembly environment | |
| US7283661B2 (en) | Image processing apparatus | |
| EP1676679A2 (en) | Robot controller for correcting a target position of a robot hand by means of a visual sensor | |
| US20140277731A1 (en) | Robot picking system, control device, and method of manufacturing a workpiece | |
| US10740649B2 (en) | Object attitude detection device, control device, and robot system | |
| CN109493389B (en) | Camera calibration method and system based on deep learning | |
| US20170151672A1 (en) | Workpiece position/posture calculation system and handling system | |
| CN111964680B (en) | Real-time positioning method of inspection robot | |
| KR101842286B1 (en) | Method for Automatic Calibration of Robot | |
| US20220402131A1 (en) | System and method for error correction and compensation for 3d eye-to-hand coordinaton | |
| CN115519533A (en) | Robot eye calibration method and device, robot and storage medium | |
| EP4116043A2 (en) | System and method for error correction and compensation for 3d eye-to-hand coordination | |
| KR102561103B1 (en) | Robot calibration system and calibration method thereof | |
| CN111546348A (en) | Robot position calibration method and position calibration system | |
| CN114677429B (en) | Positioning method and device of manipulator, computer equipment and storage medium | |
| JP2018074479A (en) | Imaging apparatus and mobile object system | |
| CN115847426A (en) | Robot motion control method, device, electronic equipment and storage medium | |
| US20230032421A1 (en) | Robot system | |
| CN113240670A (en) | Image segmentation method for object to be operated in live-wire operation scene | |
| EP4013701A1 (en) | Method and apparatus for determining object location | |
| CN114800520B (en) | High-precision hand-eye calibration method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |