WO2018214147A1

WO2018214147A1 - Robot calibration method and system, robot and storage medium

Info

Publication number: WO2018214147A1
Application number: PCT/CN2017/086094
Authority: WO
Inventors: 阳光
Original assignee: 深圳配天智能技术研究院有限公司
Priority date: 2017-05-26
Filing date: 2017-05-26
Publication date: 2018-11-29
Also published as: CN109952176B; CN109952176A

Abstract

Provided are a robot calibration method and system, a robot and a storage medium, the calibration method comprising controlling a frond end of the robot to move according to a pre-set rule (S101); detecting the change condition of a grayscale value caused by the movement, and recording a movement distance of the front end (S102); calculating a coordinate of an original position of the front end in a visual coordinate system and a coordinate of the calibration point in a robot coordinate system (S103); and according to the coordinate of the original position in the visual coordinate system and the coordinate of the calibration point in the robot coordinate system, calculating and obtaining a conversion relationship between the robot coordinate system and the visual coordinate system (S104). The precision of the conversion relationship between the robot coordinate system and the visual coordinate system can be improved by means of the method.

Description

Robot calibration method, system, robot and storage medium

【技术领域】[Technical Field]

The present invention relates to the field of robot coordinate setting technology, and in particular, to a robot calibration method, system, robot, and storage medium.

【背景技术】【Background technique】

At present, industrial robots have played an increasingly important role in production in manufacturing industries around the world. In order to make industrial robots more capable of more complex work, robots not only need better control systems, but also need to be more aware of environmental changes. Among them, robot vision has become the most important robot sensing device with its large amount of information and high information integrity. For example, in the welding process, the robot can perform precise welding of the electronic components of the circuit board. During the welding process, the robot can position the workpiece or the working surface by using the camera in the vision system, and calculate the relative position of the working scene relative to the robot to assist the robot to complete the work.

The use of the vision system to calculate the conversion relationship between the visual coordinate system and the robot coordinate system has become an important research topic for the development of robots. Obtaining the conversion relationship between the more accurate coordinate systems is the premise for solving the high-precision operation of the robot. However, existing robots are usually manually controlled to move the robot to the calibration point of the vision system so that the robot uses the vision system to calculate the conversion relationship between the visual coordinate system and the robot coordinate system. This practice usually has large human error. The error resulting in the resulting conversion relationship is large.

【发明内容】 [Summary of the Invention]

The object of the present invention is to provide a robot calibration method, system, robot and storage medium, which can improve the accuracy of the conversion relationship between the robot coordinate system and the visual coordinates.

To achieve the above object, the present invention provides a robot calibration method, the method comprising:

Controlling a front end of the robot to move from an initial position on a calibration plate according to a preset rule, wherein the calibration plate is provided with a calibration point for calibration, and a first boundary and a second boundary determined according to the calibration point;

Acquiring a plurality of images of the front end collected by the visual system during the moving process, detecting changes of gray values on both sides of the first boundary and the second boundary from the plurality of images, and recording the front end Moving distance

Calculating coordinates of the initial position of the front end in the visual coordinate system and coordinates of the calibration point in the robot coordinate system according to the change of the gray value and the moving distance;

Calculating a conversion relationship between the robot coordinate system and the visual coordinate system according to the initial position and the position of the calibration point, thereby completing calibration of the robot.

The controlling the front end of the robot to move from the initial position according to a preset rule on the calibration board includes:

Moving the front end from the initial position along the y direction of the vision system;

And moving the front end from the initial position along the x direction of the vision system.

The plurality of images of the front end acquired by the visual system are acquired during the moving process, and the change of the gray values on both sides of the first boundary and the second boundary is detected from the plurality of images. And record the moving distance of the front end including:

Acquiring the plurality of images in the moving the front end from the initial position in the y direction of the vision system, and detecting the two sides of the first boundary from the plurality of images a change in the gray value; when the gray value on both sides of the first boundary changes, the first moving distance of the front end is recorded, and the front end is obtained before and after the change of the first gray value is obtained. a first position and a second position in the direction, and a first distance between the initial position of the front end and the first boundary in the y direction;

Acquiring the plurality of images in the moving the front end from the initial position along the x direction of the vision system, and detecting the two sides of the second boundary from the plurality of images a change of the two gray values; when the gray value on both sides of the second boundary changes, the second moving distance of the front end is recorded; and the front end is at x before and after the change of the second gray value is obtained a third position and a fourth position in the direction, and a second distance between the initial position of the front end and the second boundary in the x direction.

The controlling the movement of the front end of the robot on the calibration board according to the preset rule further includes:

Moving the front end from the initial position in the x direction of the vision system to the fourth position according to a second moving distance, and then moving the front end from the fourth position along the y direction of the vision system ;

And moving the front end from the initial position in the y direction of the vision system to the second position according to the first moving distance, and then moving the front end from the second position along the x direction of the vision system mobile.

The plurality of images of the front end acquired by the visual system are acquired during the moving process, and the change of the gray values on both sides of the first boundary and the second boundary is detected from the plurality of images. And record the distance traveled by the front end, including:

Acquiring the plurality of images in the moving the front end from the fourth position along the y direction of the vision system, and detecting two sides of the first boundary from the plurality of images a change of the first gray value; when the gray value on both sides of the first boundary changes, the fifth bit and the first end of the front end in the y direction are obtained before and after the change of the first gray value Six positions;

Acquiring the plurality of images in the process of moving the front end from the second position along the x direction of the vision system, and detecting two sides of the second boundary from the plurality of images a change of the second gray value; when the gray value on both sides of the second boundary changes, the seventh bit and the first end of the front end in the x direction are obtained before and after the change of the second gray value Eight positions.

The calculating, according to the change of the gray value and the moving distance, the coordinates of the initial position of the front end in the visual coordinate system and the coordinates of the calibration point in the robot coordinate system, including:

Calculating, according to the first distance and the second distance, coordinates of an initial position of the front end in the visual coordinate system;

Obtaining a first reference line according to the first position, the second position, the fifth position, and the sixth position, and obtaining a second reference line according to the third position, the fourth position, the seventh position, and the eighth position, The coordinates of the intersection of the first reference line and the second reference line are the coordinates of the calibration point in the robot coordinate system.

The first reference line is obtained according to the first location, the second location, the fifth location, and the sixth location, specifically:

Connecting a midpoint between the first location and the second location to a midpoint between the fifth location and the sixth location to obtain a first reference line;

Obtaining a second reference line according to the third location, the fourth location, the seventh location, and the eighth location, specifically:

A midpoint between the third position and the fourth position is connected to a midpoint between the seventh position and the eighth position to obtain a second reference line.

Wherein the origin of the vision system coincides with the calibration point;

Among the two boundary lines, one boundary line closest to the x-axis of the vision system is defined as a first boundary, and one boundary line closest to the y-axis of the vision system is defined as a second boundary.

Wherein the calibration board is a checkerboard set between the black grid and the white grid, and the calibration point set on the calibration board is an intersection of the black grid and the white grid;

The first boundary and the second boundary determined according to the calibration point include:

The edge lines of the two black and white cells passing through the calibration point are taken as the first boundary line and the second boundary line, respectively.

The detecting the change of the gray value on both sides of the first boundary and the second boundary is specifically:

A change in a gray value of a plurality of pixels on both sides of the first boundary and the second boundary is detected.

In another aspect, the present invention provides a storage medium in which program data is stored, the stored data being executable to implement the robot calibration method described above.

In another aspect, the present invention provides a robot calibration system, the system comprising: a visual device, a processor, and a memory; wherein the visual device and the memory are respectively connected to the processor;

The visual device is configured to photograph a space operated by a front end of the robot to establish a corresponding visual system;

The memory is configured to store a preset movement rule of a front end of the robot, and an execution instruction of the processor;

The processor is configured to perform the following actions:

The controlling the execution of the front end of the robot on the calibration board to move from the initial position according to a preset rule, including:

The plurality of images of the front end acquired by the visual system acquired during the moving process detect a change of gray values on both sides of the first boundary and the second boundary from the plurality of images; And record the moving distance of the front end, including:

Acquiring the plurality of images in the moving the front end from the initial position in the y direction of the vision system, and detecting the two sides of the first boundary from the plurality of images a change of a gray value; when the gray value on both sides of the first boundary changes, recording a first moving distance of the front end, and obtaining a front end before and after the change of the first gray value a first position and a second position in the y direction, and a first distance between the initial position of the front end and the first boundary in the y direction;

The controlling the execution of the front end of the robot on the calibration board to move from the initial position according to a preset rule, further comprising:

The plurality of images of the front end acquired by the visual system are acquired by the processor during the moving process, and the first boundary and the second boundary are detected from the plurality of images. The change of the gray value, and record the moving distance of the front end, further including:

The calculation performed by the processor according to the change of the gray value and the moving distance, the coordinates of the initial position of the front end in the visual coordinate system and the calibration point in the robot coordinate system. Coordinates, including:

Calculating, according to the first distance and the second distance, coordinates of an initial position of the front end in the vision system;

Obtaining a first reference line according to the first position, the second position, the fifth position, and the sixth position, and obtaining a second reference line according to the third position, the fourth position, the seventh position, and the eighth position, The intersection of the first reference line and the second reference line is the coordinate of the calibration point in the robot coordinate system.

Wherein the origin of the vision system coincides with the calibration point;

In another aspect, the present invention provides a robot including a robot body and a robot calibration system,

The robot includes a front end of a robot for grasping an object;

The robot calibration system includes:

a visual device, a processor, and a memory; wherein the visual device and the memory are respectively coupled to the processor;

The processor is configured to perform the following actions:

Wherein the origin of the vision system coincides with the calibration point;

Advantageous Effects: Different from the prior art, the present invention detects the change of the gray value caused by the above movement by controlling the front end of the robot according to the preset rule, and records the moving distance of the front end; and calculates the initial of the front end. The coordinates of the position in the visual coordinate system and the coordinates of the calibration point in the robot coordinate system; the robot coordinate system is calculated according to the coordinates of the initial position in the visual coordinate system and the coordinates of the calibration point in the robot coordinate system A conversion relationship with the visual coordinate system. With the above method, the coordinates of the initial position of the front end in the visual coordinate system and the coordinates of the calibration point in the robot coordinate system can be calculated, since the coordinates of the initial position in the robot coordinate system and the calibration point are in the visual coordinate system. The coordinates below are known, so the coordinates of the initial position in the visual coordinate system and the coordinates of the initial position in the robot coordinate system, and the coordinates of the calibration point in the visual coordinate system and the calibration point in the robot coordinate system can be utilized. The coordinates are used to calculate the conversion relationship between the robot coordinate system and the visual coordinates. According to the above method, the accuracy of the conversion relationship between the robot coordinate system and the visual coordinates can be improved.

【附图说明】 [Description of the Drawings]

1 is a schematic flow chart of a first embodiment of a robot calibration method according to the present invention;

2 is a schematic flow chart of step S101 in the first embodiment of the method shown in FIG. 1;

3 is a schematic flow chart of step S102 in the first embodiment of the method shown in FIG. 1;

4 is a schematic flow chart of step S101 in the first embodiment of the method shown in FIG. 1;

FIG. 5 is a schematic flowchart of step S102 in the first embodiment of the method shown in FIG. 1;

FIG. 6 is a schematic flowchart of step S103 in the first embodiment of the method shown in FIG. 1;

7a-7h are another flow diagrams of the first embodiment of the robot calibration method of the present invention;

8 is a schematic structural diagram of an embodiment of a storage medium of the present invention;

Figure 9 is a functional block diagram of a first embodiment of the robot calibration system of the present invention;

Figure 10 is a schematic view showing the structure of an embodiment of the robot of the present invention.

【具体实施方式】【detailed description】

In order to enable those skilled in the art to better understand the technical solutions of the present invention, a robot calibration method, system, robot and storage medium provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It will be apparent that the described embodiments are merely a partial embodiment of a robot calibration method, system, robot, and storage medium of the present invention, rather than all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1, FIG. 1 is a schematic flowchart of a first embodiment of a robot calibration method according to the present invention. The method includes the following steps:

S101. The front end of the control robot moves from the initial position according to a preset rule on the calibration plate, wherein the calibration plate is provided with a calibration point for calibration, and a first boundary and a second boundary determined according to the calibration point.

Before calibrating the robot, set the calibration plate and set the calibration point on the calibration plate to calibrate the robot. The calibration plate is set within the visual range of the vision system, so that the calibration plate is photographed during the process of moving the robot to obtain a corresponding image for subsequent analysis and calculation of the image. It can be understood that the robot in this embodiment is an industrial robot, and the calibration of the robot is the calibration of the front end of the robot, and the mobile robot refers to the front end of the mobile robot.

In this embodiment, the calibration plate set by the black grid and the white grid is taken as an example. The calibration point in the visual system is the intersection of the black grid and the white grid, and the first boundary and the second boundary are the calibration points. The edge lines of two black and white grids, at this time, the gray values on both sides of the first boundary and the second boundary are large differences, one side is the gray value of the black grid, and the other side is the white grid grayscale value. At the same time, the coordinate system of the vision system is known, and its x-axis and y-axis are also determined, so that the origin of the vision system coincides with the calibration point, and the edge lines of the two black and white grids are closest to the x-axis of the visual system. A boundary line is defined as a first boundary, and a boundary line closest to the y-axis of the visual system is defined as a second boundary.

In this embodiment, if the front end of the robot contacts the calibration plate, the movement rule of the front end is set, and the front end of the robot is caused to move correspondingly from the initial position of the placement according to the movement rule. Further, the movement rule of the front end refers to moving step by step according to the minimum moving step of the robot.

It can be understood that the coordinates of the initial position of the robot front end on the calibration plate in the robot coordinate system are known at this time, and the coordinates of the calibration point in the visual coordinate system of the visual system are also known.

S102. Acquire a plurality of images of the front end collected by the visual system during the movement of the front end of the robot in step S101, and detect changes in gray values on both sides of the first boundary and the second boundary from the plurality of images, and record the front end. The distance traveled.

During the movement of the front end of the robot, the image of the calibration plate is acquired by the vision system. Since the robot moves step by step according to its minimum step, when the image of the calibration plate is acquired, the front end can be acquired once every time the movement is performed. The image of the calibration plate is recorded, and the number of movements of the front end is recorded at the same time. The moving distance of the front end can be calculated by using the number of movements of the front end recorded and the distance of each movement.

Since the vision system needs to analyze the gray value of the acquired image, the acquired image is binarized, and the acquired image does not directly display the specific position of the front end on the calibration plate, but only The area of the gray value corresponding to the front end, which is the front end projected on the calibration plate. The front end can be regarded as a stylus. When the current end contacts the calibration plate, the front end will form a corresponding projection on the calibration plate. The gray value of the projection area is different from the gray value of other positions of the calibration plate, and the calibration plate is detected. The gray value on the change and calculation, the position of the front end on the calibration plate can be obtained.

Since the first boundary and the second boundary both pass through the calibration point, and the difference of the gray values on both sides is significant, the change of the gray value on both sides of the first boundary and the second boundary is detected from the acquired image, and then The first boundary and the two sides of the second boundary respectively obtain a plurality of positions close to the calibration point, and then step S103 is performed.

S103. Calculate coordinates of the initial position of the front end in the visual coordinate system and coordinates of the calibration point in the robot coordinate system according to the change of the gray value and the moving distance.

Step S102 is capable of acquiring a plurality of positions close to the calibration point, and the initial distances of the front ends of the robots are respectively separated from the first boundary and the second boundary by a first distance and a second distance; and then the calculation may be performed according to the positions close to the calibration point. The coordinates of the calibration point in the robot coordinate system. Theoretically, the difference between the coordinates of the calibration point in the robot coordinate system and the coordinates of the set calibration point in the visual coordinate system is less than or equal to the minimum moving step of the robot, and in the ideal state, the calibration point is in the robot coordinate system. The coordinates below are the same as the coordinates of the calibration point in the visual coordinate system.

S104. Calculate a conversion relationship between the robot coordinate system and the visual coordinate system according to the initial position and the position of the calibration point, thereby completing calibration of the robot.

In step S103, the coordinates of the initial position of the front end in the visual coordinate system and the coordinates of the calibration point in the robot coordinate system are calculated, and the coordinates of the initial position of the front end in the robot coordinate system and the coordinates of the calibration point in the visual coordinate system are calculated. It is known, therefore, the coordinates of the initial position of the front end in the visual coordinate system and the initial position of the front end in the robot coordinate system, and the coordinates of the calibration point in the robot coordinate system and the calibration point in the visual coordinate system. The coordinates below, based on the matrix equation to establish the mapping relationship, can be calculated to obtain the conversion relationship between the robot coordinate system and the visual coordinate system, thus completing the calibration of the robot.

Further, as shown in FIG. 2, step S101 includes the following steps:

S1011, moving the front end from the initial position in the y direction of the vision system.

The control front end moves along the y direction of the vision system. It will be appreciated that the path of movement of the front end is parallel to the y-axis of the vision system, but not necessarily parallel to the second boundary of the set. When the calibration plate is set, the calibration plate may have an angle with the vision system, which causes the second boundary to be non-parallel to the y direction of the vision system, thereby causing the front end to be non-parallel to the second boundary when moving in the y direction of the vision system. .

S1012, moving the front end from the initial position along the x direction of the vision system.

Similar to step S1011, the moving path of the front end of the robot is parallel to the visually apparent x-axis, but not necessarily parallel to the set first boundary. When setting the calibration plate, the calibration plate may have an angle with the vision system, causing the first boundary to be non-parallel to the x-direction of the vision system, thereby causing the front end to be non-parallel to the first boundary when moving along the x-direction of the vision system. .

It should be noted that step S1011 and step S1012 do not have a strict execution order, and step S011 may be performed first, and then step S1012 may be performed; step S1012 may be performed first, and then step S1011 may be performed. The execution of step S1011 and step S1012 does not affect the final calculation result.

Further, as shown in FIG. 3, step S102 includes the following steps:

S1021: Acquire a plurality of images, and detect a change of the first gray value on both sides of the first boundary from the plurality of images; and record a first moving distance of the front end when the gray value on both sides of the first boundary changes And obtaining a first position and a second position of the front end in the y direction before and after the change of the first gray value, and a first distance between the initial position of the front end and the first boundary in the y direction.

It is to be noted that this step is not performed after the step S1011 and the step S1012 are completely performed, and the step S1021 is executed in the process of moving the front end from the initial position in the y direction of the visual system in step S1011.

That is, in the process of moving the front end from the initial position in the y direction of the visual system in step S1011, the image of the calibration plate is acquired once every time the front end is moved, and the change of the gray value on both sides of the first boundary is detected. When it is detected that the gray value on both sides of the first boundary changes, the first moving distance of the front end at this time is recorded, and the first position and the second position of the front end in the y direction before and after the change of the first gray value are obtained. And a first distance between the initial position of the front end and the first boundary in the y direction.

Specifically, in this embodiment, the calculation is performed by detecting a change in the gray value of a plurality of pixels on both sides of the first boundary. When the gray value of several pixels on both sides of the first boundary is detected, the recording is performed. At this time, the number of movements of the front end can be obtained according to the number of times and the minimum moving step of the front end. When the current end moves from one side of the first boundary to the other side during a certain movement, causing the first gray value to change, the first position of the front end in the y direction is caused before and after the change of the first gray value. For the position before the move, the second position is the position after the move. The first location and the second location are both the same side of the second boundary. By obtaining the gray value of the pixels on both sides of the first boundary, the distance between the first position and the first boundary can be calculated, and then the front end in the y direction can be calculated according to the number of movements when the front end moves to the first position. The first distance between the initial position and the first boundary.

S1022: Acquire a plurality of images, and detect a change of the second gray value on both sides of the second boundary from the plurality of images; and record a second moving distance of the front end when the gray value on both sides of the second boundary changes And obtaining a third position and a fourth position of the front end in the x direction before and after the change of the second gray value, and a second distance between the initial position of the front end and the second boundary in the x direction.

It is to be noted that this step is not performed after the step S1011 and the step S1012 are completely performed, and the step S1022 is executed in the process of moving the front end from the initial position in the x direction of the visual system in step S1012.

That is, in this step, in the process of moving the front end from the initial position in the x direction of the visual system in step S1012, the image of the calibration plate is acquired once every time the front end of the robot is moved, and the change of the gray value on both sides of the second boundary is detected. In the case, when it is detected that the gray value on both sides of the second boundary changes, the second moving distance of the front end is recorded, and the third position and the front end in the x direction are obtained before and after the change of the second gray value. Four positions, and a second distance between the initial position of the front end and the second boundary in the x direction.

Specifically, in this embodiment, the calculation is performed by detecting a change in the gray value of a plurality of pixels on both sides of the second boundary. When the gray value of several pixels on both sides of the second boundary is detected to be changed, the recording is performed. At this time, the number of movements of the front end can be obtained according to the number of times and the minimum moving step of the front end. When the current end moves from one side of the second boundary to the other side during a certain movement, causing the second gray value to change, the third position of the front end in the x direction is caused before and after the change of the second gray value. For the position before the move, the fourth position is the position after the move. The third location and the fourth location are both the same side of the first boundary. By obtaining the gray value of the pixels on both sides of the second boundary, the distance between the third position and the second boundary can be calculated, and then the front end in the x direction can be calculated according to the number of movements when the front end moves to the third position. The second distance between the initial position and the second boundary.

Further, as shown in FIG. 4, step S101 further includes the following steps:

S1013. Move the front end from the initial position along the x-direction of the visual system to the fourth position according to the second moving distance, and then move the front end from the fourth position along the y direction of the visual system.

Controlling the front end of the robot according to the second distance, moving from the initial position on the calibration plate in the x direction along the x direction of the vision system until moving to the fourth position obtained in step S1022, and then moving the front end from the fourth position along the vision The y direction of the system moves. The specific step of moving the front end from the fourth position in the y direction of the vision system is similar to step S1011.

S1014. Move the front end from the initial position along the y direction of the vision system to the second position according to the first moving distance, and then move the front end from the second position along the x direction of the vision system.

Controlling the front end of the robot according to the first distance, moving from the initial position of the calibration plate in the y direction along the y direction of the vision system to the second position obtained in step S1021, and then moving the front end from the second position along the x direction of the vision system mobile. The specific step of moving the front end from the second position in the x direction of the vision system is similar to step S1012.

Further, as shown in FIG. 5, step S102 further includes the following steps:

S1023, in the process of moving the front end from the fourth position in the y direction of the visual system in step S1013, acquiring a plurality of images, and detecting a change of the first gray value on both sides of the first boundary from the plurality of images; When the gradation value on both sides of the first boundary changes, the fifth and sixth positions of the front end in the y direction before and after the change of the first gradation value are obtained.

It should be noted that this step is performed during the execution of step S1013.

That is, in this step, in the process of moving the front end from the fourth position in the y direction of the visual system in step S1013, the image of the calibration plate is acquired once for each movement of the front end of the robot. When the current end moves from one side of the first boundary to the other side at a certain movement, causing the first gradation value to change, the fifth position of the front end in the y direction is caused before and after the change of the first gradation value. For the position before the movement, the sixth position is the position after the movement, and the distance between the fifth position and the sixth position is the minimum moving step of the front end, and the fifth position and the sixth position are respectively Both sides of the first boundary.

S1024, in the process of moving the front end from the second position along the x direction of the visual system in step S1014, acquiring a plurality of images, and detecting a change of the second gray value on both sides of the second boundary from the plurality of images; When the gradation value on both sides of the second boundary is changed, the seventh and eighth positions of the front end in the x direction before and after the change of the second gradation value are obtained.

It should be noted that this step is performed during the execution of step S1014.

That is, in this step, in the process of moving the front end from the second position in the x direction of the visual system in step S1014, the image of the calibration plate is acquired once for each movement of the front end of the robot. When the current end moves from one side of the second boundary to the other side at a certain movement, causing a change in the second gradation value, causing the seventh position of the front end in the x direction before and after the change of the second gradation value For the position before the movement, the eighth position is the position after the movement, and the distance between the seventh position and the eighth position is the minimum moving step of the front end, and the seventh position and the eighth position are respectively Both sides of the second boundary.

Further, as shown in FIG. 6, step S103 includes the following steps:

S1011: Calculate coordinates of the initial position of the front end in the visual coordinate system according to the first distance and the second distance.

The first distance is the distance between the front end in the y direction and the first boundary, and the second distance is the distance between the front end of the robot in the x direction and the second boundary. Since the first boundary is parallel to the x direction of the vision system and the second boundary is parallel to the y direction when setting the calibration plate, the first boundary can be regarded as the x axis and the second boundary as the y axis. The first distance can be regarded as the distance of the front end from the x-axis in the y direction, and the second distance is regarded as the distance of the front end from the y-axis in the x direction, that is, the initial position of the front end is obtained in the visual coordinate system. coordinate. It can be understood that there may be an error when setting the calibration plate, and thus the coordinates of the initial position of the obtained front end in the visual coordinate system may have errors.

S1032, obtaining a first reference line according to the first position, the second position, the fifth position, and the sixth position, and obtaining a second reference line according to the third position, the fourth position, the seventh position, and the eighth position, the first reference line The intersection with the second reference line is the coordinate of the corrected calibration point in the robot coordinate system.

The first reference line obtained according to the first position, the second position, the fifth position, and the sixth position is specifically a middle point between the first position and the second position and a middle position between the fifth position and the sixth position. Connect the points to get the first reference line. Obtaining a second reference line according to the third position, the fourth position, the seventh position, and the eighth position, specifically, performing a midpoint between the third position and the fourth position and a midpoint between the seventh position and the eighth position Connect to the second reference line. The coordinates of the intersection of the first reference line and the second reference line are the coordinates of the calibration point in the robot coordinate system. Theoretically, the difference between the coordinates of the calibration point in the robot coordinate system and the coordinates of the set calibration point in the visual coordinate system is less than or equal to the minimum moving step of the robot. In the ideal state, the calibration point is in the robot coordinate system. The coordinates below are the same as the coordinates of the calibration point in the visual coordinate system. Use the coordinates of the initial position of the front end in the visual coordinate system and the initial position of the front end in the coordinates of the robot coordinate system, and the coordinates of the calibration point in the robot coordinate system and the coordinates of the calibration point in the visual coordinate system, according to the matrix equation By establishing a mapping relationship, a conversion relationship between the robot coordinate system and the visual coordinate system can be obtained, thereby completing the calibration of the robot.

Referring to Figures 7a to 7h, a detailed description of the first embodiment of the robot calibration method of the present invention will be further described by way of example.

The calibration plate is directly opposite to the vision system. The calibration point on the calibration plate is the origin of the vision system. The first boundary coincides with the x-axis and the second boundary coincides with the y-axis. The calibration board is a checkerboard set between black and white. As shown in FIG. 7a, the area A and the area D are white spaces, and the area B and the area C are black spaces. In order to show the influence of the movement of the front end on the gray value, the black area B and C are not filled. Since the calibration plate is facing the vision system at this time, the x-axis and the y-axis are the first boundary and the second boundary, respectively, and the calibration point is the intersection of the x-axis and the y-axis in the visual system.

Figure 7a is a grayscale image of the calibration plate acquired by the vision system. The shaded area E is the projection on the calibration plate when the front end contacts the calibration plate. (It can be understood that only the projection near the front end of the calibration plate is drawn in the figure. In fact, the projection of the front end should be continuous and extended to the edge of the calibration plate.). Each cell in the figure represents a pixel. At this point, the vision system can only get the grayscale distribution on the calibration plate, and the position of the contact point where the front end is in contact with the calibration plate cannot be directly obtained from the image.

After the step S1011, the front end is moved from the initial position along the y direction of the visual system. In the process, a plurality of images are acquired through step S1021, and the first gray level on both sides of the first boundary is detected from the acquired plurality of images. The change in value. Assuming that the front end of the robot moves a minimum step, that is, from one side of the first boundary to the other side of the first boundary, the acquired image is as shown in FIG. 7b, and the calculation is based on the gray distribution of FIGS. 7a and 7b. a change of the first gray value on both sides of a boundary, thereby obtaining the position of the contact point F of the front end of the robot and the calibration plate, and the first position 10 and the second position 20 of the front end before and after the movement (ie, The position of the contact point between the front end and the calibration plate before and after the movement). At the same time, the first distance between the initial position of the front end of the robot and the first boundary in the y direction is calculated by the gray value in the pixels on both sides of the first boundary.

The front end of the robot is returned to the initial position, and the front end is moved from the initial position in the x direction of the vision system through step S1012. In the process, a plurality of images are acquired through step S1022, and detected from the acquired plurality of images. A change in the second gray value on either side of the second boundary. Assuming that the front end of the robot moves a minimum step, that is, from one side of the second boundary to the other side of the second boundary, the acquired image is as shown in FIG. 7c, and is calculated according to the gray scale distribution of FIGS. 7a and 7c. The change of the second gray value on both sides of the two boundaries further obtains the third position 30 and the fourth position 40 of the front end of the robot before and after the movement. At the same time, the second distance between the initial position of the front end and the second boundary in the x direction is calculated by the gray value in the pixels on both sides of the second boundary.

The front end of the robot is returned to the initial position, and the front end is moved from the initial position to the fourth position in the x direction according to the second moving distance, and then the front end is moved from the fourth position along the y direction of the vision system according to the second moving distance. mobile. In the process of moving the front end from the fourth position in the y direction of the visual system, a plurality of images are acquired through step S1023, and changes of the first gray value on both sides of the first boundary are detected from the acquired plurality of images. . Assuming that the front end of the robot moves a minimum step, that is, from the fourth position to the other side of the first boundary, the acquired image is as shown in FIG. 7d, according to moving the front end in the x direction from one side of the second boundary to another. The image obtained by one side (not shown) and the gray scale distribution of FIG. 7d are calculated to obtain the change of the first gray value on both sides of the first boundary, thereby obtaining the fifth position 50 and the front of the front end before and after the movement. Six positions 60.

The front end of the robot is again returned to the initial position, and the front end is moved from the initial position to the second position in the y direction according to the first moving distance, and then the front end is moved from the second position along the x direction of the vision system according to the first moving distance. mobile. In the process of moving the front end from the second position along the x direction of the vision system, a plurality of images are acquired through step S1024, and changes of the second gray value on both sides of the second boundary are detected from the acquired plurality of images. . Assuming that the front end of the robot moves a minimum step, that is, from one side of the second boundary to the other side of the second boundary, the acquired image is as shown in FIG. 7e, according to the side of the front end in the y direction from the first boundary. The image acquired by moving to the other side (not shown) and the gray-scale distribution of FIG. 7e are calculated to obtain the change of the second gray value on both sides of the second boundary, thereby obtaining the seventh position of the front end before the movement and after the movement. 70 and eighth position 80.

According to the above steps, the first distance between the initial position of the front end in the y direction and the first boundary and the second distance between the initial position of the front end and the second boundary in the x direction can calculate the initial position of the front end. The coordinates in the visual coordinate system.

As shown in FIG. 7f, according to step S1021 and step S1023, the first position 10, the second position 20, the fifth position 50, and the sixth position 60 are acquired, and the midpoint between the first position 10 and the second position 20 is obtained. And the midpoint connection between the fifth position 50 and the sixth position 60, the first reference line L1 shown in Fig. 7f is obtained. As shown in FIG. 7g, according to step S1021 and step S1023, the third position 30, the fourth position 40, the seventh position 70, and the eighth position 80 are acquired, and the midpoint between the third position 30 and the fourth position 40 is obtained. And the midpoint connection between the seventh position 70 and the eighth position 80, the second reference line L2 shown in Fig. 7g is obtained. As shown in FIG. 7h, the first reference line L1 and the second reference line L2 intersect, and the coordinates of the intersection point H are the coordinates of the calibration point in the robot coordinate system.

Since the change of the gray value on both sides of the first boundary or the second boundary is caused by the front end of the robot at the minimum moving step of the movement, the distance between the first position and the second position is obtained, and the third The distance between the position and the fourth position, the distance between the fifth position and the sixth position, and the distance between the seventh position and the eighth position are substantially equal to the minimum moving step of the front end, so in theory, the standard The difference between the coordinates of the fixed point in the robot coordinate system and the coordinates of the set calibration point in the visual coordinate system is less than or equal to the minimum moving step of the robot. In the ideal state, the coordinates and the standard of the calibration point in the robot coordinate system The coordinates of the fixed point in the visual coordinate system are the same. Then use the coordinates of the initial position of the front end in the visual coordinate system and the initial position of the front end in the coordinate of the robot coordinate system, and the coordinates of the calibration point in the robot coordinate system and the coordinates of the calibration point in the visual coordinate system, according to the matrix. By establishing a mapping relationship, the conversion relationship between the robot coordinate system and the visual coordinate system can be obtained, thereby completing the calibration of the robot.

Referring to FIG. 8, FIG. 8 is a schematic structural diagram of an embodiment of a storage medium of the present invention. As shown in FIG. 8, the storage medium 400 in the present embodiment stores program data 401 that can be executed, and the program data 401 is executed to implement the robot calibration method embodiment shown in FIGS. 1 to 6. In this embodiment, the storage medium 400 may be a storage module of the smart terminal, a mobile storage device (such as a mobile hard disk, a USB flash drive, etc.), a network cloud disk, or an application storage platform, and the like.

Referring to Figure 9, Figure 9 is a functional block diagram of a first embodiment of the robot calibration system of the present invention. The robot calibration system 200 includes a vision device 201, a processor 202, and a memory 204; wherein the vision device 201 and the memory 204 are respectively coupled to the processor 202;

The visual device 201 is configured to photograph a space operated by the front end of the robot to establish a corresponding visual system. The memory 204 is configured to store a preset movement rule of the front end and an execution instruction of the processor.

The processor 202 is configured to perform the following actions:

Controlling a front end of the robot to move from an initial position on a calibration plate according to a preset rule, wherein the calibration plate is provided with a calibration point for calibration, and a first boundary and a second boundary determined according to the calibration point; Acquiring a plurality of images of the front end collected by the visual system during the moving process, detecting changes of gray values on both sides of the first boundary and the second boundary from the plurality of images, and recording the front end a moving distance; calculating a coordinate of the initial position of the front end in the visual coordinate system and a coordinate of the calibration point in the robot coordinate system according to the change of the gray value and the moving distance; according to the initial The position and the position of the calibration point are calculated to obtain a conversion relationship between the robot coordinate system and the visual coordinate system, thereby completing calibration of the robot.

In this embodiment, the action performed by the processor 202 corresponds to the robot calibration method shown in FIG. 1 to FIG. 6, and the detailed description of the robot calibration method is specifically referred to, and details are not described herein again.

Referring to FIG. 10, FIG. 10 is a schematic structural view of an embodiment of a robot according to the present invention. The robot 300 includes a robot body 301 and a robot calibration system 302.

The robot 301 includes a front end of a robot for grabbing an object. The robot calibration system 302 includes a vision device, a processor, and a memory; wherein the vision device and the memory are respectively coupled to the processor. A visual device for photographing the space of the front end operation to establish a corresponding vision system. A memory for storing preset movement rules of the front end and execution instructions of the processor. A processor that performs the following actions:

The robot calibration system in this embodiment is the same as the robot calibration system shown in FIG. 9, and details are not described herein again.

The above is only the embodiment of the present invention, and is not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformation made by the specification and the drawings of the present invention may be directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims

A robot calibration method includes:

Controlling a front end of the robot to move from an initial position on a calibration plate according to a preset rule, wherein the calibration plate is provided with a calibration point for calibration, and a first boundary and a second boundary determined according to the calibration point;

Acquiring a plurality of images of the front end collected by the visual system during the moving process, detecting changes of gray values on both sides of the first boundary and the second boundary from the plurality of images, and recording the front end Moving distance

Calculating coordinates of the initial position of the front end in the visual coordinate system and coordinates of the calibration point in the robot coordinate system according to the change of the gray value and the moving distance;

Calculating a conversion relationship between the robot coordinate system and the visual coordinate system according to the initial position and the position of the calibration point, thereby completing calibration of the robot.
The calibration method according to claim 1, wherein the controlling the front end of the robot to move from the initial position according to a preset rule on the calibration board comprises:

Moving the front end from the initial position along the y direction of the vision system;

And moving the front end from the initial position along the x direction of the vision system.
The calibration method according to claim 2, wherein said acquiring a plurality of images of said front end acquired by a vision system during said moving, detecting said first boundary and said second from said plurality of images The change of the gray value on both sides of the boundary, and recording the moving distance of the front end includes:

Acquiring the plurality of images in the moving the front end from the initial position in the y direction of the vision system, and detecting the two sides of the first boundary from the plurality of images a change in the gray value; when the gray value on both sides of the first boundary changes, the first moving distance of the front end is recorded, and the front end is obtained before and after the change of the first gray value is obtained. a first position and a second position in the direction, and a first distance between the initial position of the front end and the first boundary in the y direction;

Acquiring the plurality of images in the moving the front end from the initial position along the x direction of the vision system, and detecting the two sides of the second boundary from the plurality of images a change of the two gray values; when the gray value on both sides of the second boundary changes, the second moving distance of the front end is recorded; and the front end is at x before and after the change of the second gray value is obtained a third position and a fourth position in the direction, and a second distance between the initial position of the front end and the second boundary in the x direction.
The calibration method according to claim 3, wherein the controlling the front end of the robot to move according to a preset rule on the calibration plate further comprises:

Moving the front end from the initial position in the x direction of the vision system to the fourth position according to a second moving distance, and then moving the front end from the fourth position along the y direction of the vision system ;

And moving the front end from the initial position in the y direction of the vision system to the second position according to the first moving distance, and then moving the front end from the second position along the x direction of the vision system mobile.
The calibration method according to claim 4, wherein said acquiring a plurality of images of said front end acquired by a vision system during said moving, detecting said first boundary and said second from said plurality of images The change of the gray value on both sides of the boundary, and record the moving distance of the front end, also includes:

Acquiring the plurality of images in the moving the front end from the fourth position along the y direction of the vision system, and detecting two sides of the first boundary from the plurality of images a change of the first gray value; when the gray value on both sides of the first boundary changes, the fifth bit and the first end of the front end in the y direction are obtained before and after the change of the first gray value Six positions;

Acquiring the plurality of images in the process of moving the front end from the second position along the x direction of the vision system, and detecting two sides of the second boundary from the plurality of images a change of the second gray value; when the gray value on both sides of the second boundary changes, the seventh bit and the first end of the front end in the x direction are obtained before and after the change of the second gray value Eight positions.
The calibration method according to claim 5, wherein said calculating a coordinate of said initial position of said front end in a visual coordinate system and said calibration point in said robot based on said change in said gray value and said moving distance Coordinates in the coordinate system, including:

Calculating, according to the first distance and the second distance, coordinates of an initial position of the front end in the visual coordinate system;

Obtaining a first reference line according to the first position, the second position, the fifth position, and the sixth position, and obtaining a second reference line according to the third position, the fourth position, the seventh position, and the eighth position, The coordinates of the intersection of the first reference line and the second reference line are the coordinates of the calibration point in the robot coordinate system.
The calibration method according to claim 6, wherein the first reference line is obtained according to the first position, the second position, the fifth position, and the sixth position, specifically:

Connecting a midpoint between the first location and the second location to a midpoint between the fifth location and the sixth location to obtain a first reference line;

Obtaining a second reference line according to the third location, the fourth location, the seventh location, and the eighth location, specifically:

A midpoint between the third position and the fourth position is connected to a midpoint between the seventh position and the eighth position to obtain a second reference line.
The calibration method according to claim 7, wherein an origin of the vision system coincides with the calibration point;

Among the two boundary lines, one boundary line closest to the x-axis of the vision system is defined as a first boundary, and one boundary line closest to the y-axis of the vision system is defined as a second boundary.
The calibration method according to claim 1, wherein the calibration plate is a checkerboard set between black and white grids, and a calibration point set on the calibration plate is an intersection of the black grid and the white grid;

The first boundary and the second boundary determined according to the calibration point include:

The edge lines of the two black and white cells passing through the calibration point are taken as the first boundary line and the second boundary line, respectively.
The calibration method according to claim 1, wherein the detecting the change of the gray value on both sides of the first boundary and the second boundary is specifically:

A change in a gray value of a plurality of pixels on both sides of the first boundary and the second boundary is detected.
A storage medium, wherein the storage medium stores program data, the stored data being executable to implement the robot calibration method according to any one of claims 1-10.
A robot calibration system, comprising: a visual device, a processor, and a memory; wherein the visual device and the memory are respectively connected to the processor;

The visual device is configured to photograph a space operated by a front end of the robot to establish a corresponding visual system;

The memory is configured to store a preset movement rule of a front end of the robot, and an execution instruction of the processor;

The processor is configured to perform the following actions:

Controlling a front end of the robot to move from an initial position on a calibration plate according to a preset rule, wherein the calibration plate is provided with a calibration point for calibration, and a first boundary and a second boundary determined according to the calibration point;

Acquiring a plurality of images of the front end collected by the visual system during the moving process, detecting changes of gray values on both sides of the first boundary and the second boundary from the plurality of images, and recording the front end Moving distance

Calculating coordinates of the initial position of the front end in the visual coordinate system and coordinates of the calibration point in the robot coordinate system according to the change of the gray value and the moving distance;

Calculating a conversion relationship between the robot coordinate system and the visual coordinate system according to the initial position and the position of the calibration point, thereby completing calibration of the robot.
The calibration system according to claim 12, wherein said controlling said front end of said robot to move from said initial position on said calibration plate according to a preset rule comprises:

Moving the front end from the initial position along the y direction of the vision system;

And moving the front end from the initial position along the x direction of the vision system.
The calibration system according to claim 13, wherein said acquiring a plurality of images of said front end acquired by a vision system during said moving process detects said first boundary and said second boundary from said plurality of images The change of the gray value on both sides; and record the moving distance of the front end, including:

Acquiring the plurality of images in the moving the front end from the initial position in the y direction of the vision system, and detecting the two sides of the first boundary from the plurality of images a change of a gray value; when the gray value on both sides of the first boundary changes, recording a first moving distance of the front end, and obtaining a front end before and after the change of the first gray value a first position and a second position in the y direction, and a first distance between the initial position of the front end and the first boundary in the y direction;

Acquiring the plurality of images in the moving the front end from the initial position along the x direction of the vision system, and detecting the two sides of the second boundary from the plurality of images a change of the two gray values; when the gray value on both sides of the second boundary changes, the second moving distance of the front end is recorded; and the front end is at x before and after the change of the second gray value is obtained a third position and a fourth position in the direction, and a second distance between the initial position of the front end and the second boundary in the x direction.
The calibration system according to claim 14, wherein the controlling the execution of the front end of the robot on the calibration plate to move from the initial position according to a preset rule further comprises:

Moving the front end from the initial position in the x direction of the vision system to the fourth position according to a second moving distance, and then moving the front end from the fourth position along the y direction of the vision system ;

And moving the front end from the initial position in the y direction of the vision system to the second position according to the first moving distance, and then moving the front end from the second position along the x direction of the vision system mobile.
The calibration system according to claim 15, wherein said plurality of images of said front end acquired by said visual system are acquired by said processor during said moving, said detecting said said plurality of images The change of the gray value on both sides of the first boundary and the second boundary, and recording the moving distance of the front end, further including:

Acquiring the plurality of images in the moving the front end from the fourth position along the y direction of the vision system, and detecting two sides of the first boundary from the plurality of images a change of the first gray value; when the gray value on both sides of the first boundary changes, the fifth bit and the first end of the front end in the y direction are obtained before and after the change of the first gray value Six positions;

Acquiring the plurality of images in the process of moving the front end from the second position along the x direction of the vision system, and detecting two sides of the second boundary from the plurality of images a change of the second gray value; when the gray value on both sides of the second boundary changes, the seventh bit and the first end of the front end in the x direction are obtained before and after the change of the second gray value Eight positions.
The calibration system according to claim 16, wherein said calculating, by said processor, said coordinate of said initial position of said front end in a visual coordinate system based on said change in said gray value and said moving distance The coordinates of the calibration point in the robot coordinate system include:

Calculating, according to the first distance and the second distance, coordinates of an initial position of the front end in the vision system;

Obtaining a first reference line according to the first position, the second position, the fifth position, and the sixth position, and obtaining a second reference line according to the third position, the fourth position, the seventh position, and the eighth position, The intersection of the first reference line and the second reference line is the coordinate of the calibration point in the robot coordinate system.
The calibration system according to claim 17, wherein the first reference line is obtained according to the first position, the second position, the fifth position, and the sixth position, specifically:

Connecting a midpoint between the first location and the second location to a midpoint between the fifth location and the sixth location to obtain a first reference line;

Obtaining a second reference line according to the third location, the fourth location, the seventh location, and the eighth location, specifically:

A midpoint between the third position and the fourth position is connected to a midpoint between the seventh position and the eighth position to obtain a second reference line.
The calibration method according to claim 18, wherein the corrected reference point is an intersection of the first reference lower and the second reference line.
The calibration method according to claim 19, wherein an origin of said vision system coincides with said calibration point;

Among the two boundary lines, one boundary line closest to the x-axis of the vision system is defined as a first boundary, and one boundary line closest to the y-axis of the vision system is defined as a second boundary.
The calibration system according to claim 12, wherein the calibration plate is a checkerboard set between black and white grids, and a calibration point set on the calibration plate is an intersection of the black grid and the white grid;

The first boundary and the second boundary determined according to the calibration point include:

The edge lines of the two black and white cells passing through the calibration point are taken as the first boundary line and the second boundary line, respectively.
a robot including a robot body and a robot calibration system,

The robot includes a front end of a robot for grasping an object;

The robot calibration system includes:

a visual device, a processor, and a memory; wherein the visual device and the memory are respectively coupled to the processor;

The visual device is configured to photograph a space operated by a front end of the robot to establish a corresponding visual system;

The memory is configured to store a preset movement rule of a front end of the robot, and an execution instruction of the processor;

The processor is configured to perform the following actions:

Controlling a front end of the robot to move from an initial position on a calibration plate according to a preset rule, wherein the calibration plate is provided with a calibration point for calibration, and a first boundary and a second boundary determined according to the calibration point;

Acquiring a plurality of images of the front end collected by the visual system during the moving process, detecting changes of gray values on both sides of the first boundary and the second boundary from the plurality of images, and recording the front end Moving distance

Calculating coordinates of the initial position of the front end in the visual coordinate system and coordinates of the calibration point in the robot coordinate system according to the change of the gray value and the moving distance;

Calculating a conversion relationship between the robot coordinate system and the visual coordinate system according to the initial position and the position of the calibration point, thereby completing calibration of the robot.
The robot according to claim 22, wherein an origin of said vision system coincides with said calibration point;

Among the two boundary lines, one boundary line closest to the x-axis of the vision system is defined as a first boundary, and one boundary line closest to the y-axis of the vision system is defined as a second boundary.