CN111590593B - Calibration method, device and system of mechanical arm and storage medium - Google Patents
Calibration method, device and system of mechanical arm and storage medium Download PDFInfo
- Publication number
- CN111590593B CN111590593B CN202010567739.6A CN202010567739A CN111590593B CN 111590593 B CN111590593 B CN 111590593B CN 202010567739 A CN202010567739 A CN 202010567739A CN 111590593 B CN111590593 B CN 111590593B
- Authority
- CN
- China
- Prior art keywords
- calibration
- coordinate system
- position information
- calibration block
- mechanical arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Abstract
The embodiment of the invention provides a calibration method, a calibration device, a calibration system and a storage medium of a mechanical arm. The method is applied to a mechanical arm comprising a flange, wherein a calibration block is arranged at the tail end of the flange, first position information of a plurality of boundary points of an area to be calibrated is obtained, a calibration path in the range of the area to be calibrated is generated according to the first position information, and a plurality of calibration path points on the calibration path are determined; controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor; and determining a transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
Description
Technical Field
The embodiment of the invention relates to the technical field of robots, in particular to a method, a device and a system for calibrating a mechanical arm and a storage medium.
Background
As automatic control apparatuses capable of performing various kinds of work are increasingly widely used, a robot arm has been widely used in various fields as an automatic control apparatus that simulates the function of a human arm and can perform various kinds of work.
Currently, such robotic systems have multiple joints and allow motion in a plane or three-dimensional space to achieve various motions. The mechanical arm needs to be normally used, and hand-eye calibration needs to be performed before use, that is, position information of a target point obtained by a Three-dimensional (3D) visual sensor needs to be transformed to a base coordinate system of the mechanical arm, that is, a transformation relation between a 3D visual sensor coordinate system and a mechanical arm base coordinate system is solved. And then, the detection of the motion target point of the mechanical arm can be finished according to the hand-eye calibration result, and the automatic movement of the mechanical arm is finished through motion planning.
However, in the hand-eye calibration method in the related art, the 3D vision sensor needs to be installed at the end of the manipulator, the measurement workpiece is placed on the ground for measurement and calculation, the manipulator is manually moved to different positions repeatedly, and the homogeneous transformation matrix between the manipulator flange coordinate system and the 3D vision sensor coordinate system is solved through simultaneous equations of results obtained by multiple measurement and calculation. In the hand-eye calibration process in the related art, after the 3D camera collects one point, a calibration operator needs to manually move the mechanical arm to the next position, which is unfriendly and time-consuming to operate and has low calibration efficiency.
Disclosure of Invention
The embodiment of the invention provides a calibration method, a calibration device, a calibration system and a storage medium of a mechanical arm, and aims to at least solve the problem of low calibration efficiency in a hand-eye calibration process in the related art.
According to an embodiment of the invention, a calibration method of a mechanical arm is provided, which is applied to the mechanical arm comprising a flange and a calibration block arranged at the tail end of the flange, and the method comprises the following steps: acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path within the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path; controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor; and determining a transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
In at least one exemplary embodiment, the obtaining first position information of a plurality of boundary points of the area to be calibrated includes: and controlling the mechanical arm to drive the calibration block to move to the plurality of boundary points of the area to be calibrated, and respectively collecting the flange positions when the mechanical arm drives the calibration block to move to the plurality of boundary points as the first position information.
In at least one exemplary embodiment, generating a calibration path within the region to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path includes: determining a starting point and an end point in the boundary points, and generating a continuous calibration path without collision from the starting point to the end point in the range of the region to be calibrated; and selecting a plurality of calibration path points on the calibration path, and determining fourth position information of the plurality of calibration path points according to the first position information and the calibration path point selection interval.
In at least one exemplary embodiment, obtaining second position information of the calibration block under a base coordinate system of the robotic arm as the calibration block moves to each calibration path point comprises: when the mechanical arm drives the calibration block to move to each calibration path point, acquiring the flange position of the mechanical arm; and determining second position information of the calibration block under the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block relative to an origin of the flange coordinate system.
In at least one exemplary embodiment, determining second position information of the calibration block in the base coordinate system based on the flange position and a coordinate system offset distance comprises: for each calibration path point, the flange position corresponding to the current calibration path point is determined according to the following formula
And a coordinate system offset distance x0,y0,z0Determining second position information a of the calibration block in the base coordinate systemi(x,y,z)：
Wherein, c·Represents cos (. cndot.), s·Denotes sin (. cndot.), x0,y0,z0Offset distances of an origin of the calibration block coordinate system relative to an origin of the flange coordinate system under x, y and z axes are respectively, and i is 1, 2.
In at least one exemplary embodiment, the obtaining, by the vision sensor, third position information of the calibration block in the vision sensor coordinate system when the calibration block moves to each calibration path point includes: when the calibration block moves to each calibration path point, extracting the position of the central point of the calibration block on a preset plane through the visual sensor as third position information b of the calibration block in a visual sensor coordinate system when the calibration block moves to each calibration path pointi(x, y, z), i 1,2, k, k is the total number of the plurality of calibration path points.
In at least one example embodiment, determining the transformation relationship between the vision sensor coordinate system and the base coordinate system based on the second location information and the third location information comprises: according to R ═ VUTDetermining a rotation matrix R between the vision sensor coordinate system and the base coordinate system and based thereont-b-Ra determines a translation vector t between the vision sensor coordinate system and the base coordinate system,
wherein the content of the first and second substances,
airepresenting the second position information a of the calibration block under the base coordinate system when the calibration block moves to the ith calibration path pointi(x,y,z)，biThird position information b of the calibration block in the visual sensor coordinate system when the calibration block moves to the ith calibration path pointi(x, y, z), V and U are the pair matrices, respectively
According to H ═ U Λ VTA matrix obtained by matrix decomposition, T represents transposing the matrix, a'i＝ai-a，b’i＝bi-b, k is the total number of said plurality of nominal path points.
According to another embodiment of the present invention, there is provided a calibration device for a robot arm, which is applied to a robot arm including a flange and having a calibration block mounted at an end of the flange, the calibration device including: the calibration path planning module is used for acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path in the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path; the position calculation module is used for controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor; and the calibration result calculation module is set to determine the transformation relation between the visual sensor coordinate system and the base coordinate system according to the second position information and the third position information.
According to another embodiment of the invention, a calibration system of a mechanical arm is provided, which comprises a mechanical arm, a calibration block, a visual sensor and a computing device, wherein the mechanical arm comprises a flange, the calibration block is installed at the end of the flange, and wired or wireless communication connections are arranged between the mechanical arm and the computing device and between the visual sensor and the computing device; the computing device is configured to: acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path within the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path; controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor; and determining a transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
In at least one example embodiment, the calculation means is arranged to obtain first position information of a plurality of boundary points of the area to be calibrated by: and controlling the mechanical arm to drive the calibration block to move to the plurality of boundary points of the area to be calibrated, and respectively collecting the flange positions when the mechanical arm drives the calibration block to move to the plurality of boundary points as the first position information.
In at least one example embodiment, the computing device is configured to obtain second position information of the calibration block in a base coordinate system of the robotic arm as the calibration block moves to each calibration path point by: when the mechanical arm drives the calibration block to move to each calibration path point, acquiring the flange position of the mechanical arm; and determining second position information of the calibration block under the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block relative to an origin of the flange coordinate system.
In at least one example embodiment, the computing device is configured to obtain, by the visual sensor, third position information of the calibration block in the visual sensor coordinate system when the calibration block moves to each calibration path point by: when the calibration block moves to each calibration path point, extracting the position of the central point of the calibration block on a preset plane through the visual sensor as third position information b of the calibration block in a visual sensor coordinate system when the calibration block moves to each calibration path pointi(x, y, z), i 1,2, k, k is the total number of the plurality of calibration path points.
According to a further embodiment of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
According to the invention, because the calibration path is generated based on the area to be calibrated, a plurality of calibration path points on the calibration path are selected, the mechanical arm is controlled to drive the calibration block to move according to the calibration path, the second position information of the calibration block under the base coordinate system of the mechanical arm and the third position information of the calibration block under the visual sensor coordinate system when the calibration block reaches each calibration path point are obtained in the moving process, and the transformation relation between the visual sensor coordinate system and the base coordinate system is determined according to the second position information and the third position information, the problem of low calibration efficiency in the hand-eye calibration process in the related technology can be solved, the mechanical arm does not need to be manually operated in the calibration process, the mechanical arm can automatically move and collect the position information according to the planned path, the operation process is simple, and the calibration efficiency is high.
Drawings
FIG. 1 is a flow chart of a method for calibration of a robotic arm according to an embodiment of the present invention;
FIG. 2 is a block diagram of a calibration apparatus for a robot arm according to an embodiment of the present invention;
FIG. 3 is a schematic view of a calibration system for a robotic arm according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a calibration path within a region to be calibrated according to an embodiment of the invention
FIG. 5 is a flow chart of a robot arm calibration scheme according to an embodiment of the present invention.
Detailed Description
In the related technology, when a mechanical arm is calibrated, a 3D vision sensor is installed at the tail end of a mechanical arm, a measurement workpiece is placed on the ground, a homogeneous transformation matrix of a flange coordinate system at the tail end of the mechanical arm relative to a robot base coordinate system is calculated, then the mechanical arm needs to be moved manually to different positions, different three-dimensional positions (X, Y and Z) of the circle center of a circular hole of the measurement workpiece in a camera coordinate system are read through the 3D vision sensor, and then a simultaneous equation set is used for solving the homogeneous transformation matrix between the flange coordinate system of the mechanical arm and the 3D vision sensor coordinate system by adopting a least square method. However, in this scheme, each time the 3D camera finishes capturing one point, the calibration operator needs to manually move the mechanical arm to the next position, which is not friendly and time-consuming to operate, and the calibration efficiency is low.
In view of the above problems, embodiments of the present invention provide a method, an apparatus, a system, and a storage medium for calibrating a robot arm. According to the scheme, an automatic calibration path (preferably a collision-free path) is planned for the mechanical arm, the mechanical arm does not need to be operated manually in the 3D hand-eye calibration process, the mechanical arm can automatically move and acquire position information according to the planned path in advance, the operation process is simple, and the calibration efficiency is high.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings in conjunction with the embodiments.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
In the present embodiment, a calibration method applied to a robot arm including a flange and a calibration block mounted at an end of the flange is provided, and fig. 1 is a flowchart of the calibration method of the robot arm according to the embodiment of the present invention, as shown in fig. 1, the flowchart includes the following steps:
step S102, acquiring first position information of a plurality of boundary points of a to-be-calibrated area, generating a calibration path within the range of the to-be-calibrated area according to the first position information, and determining a plurality of calibration path points on the calibration path, wherein the to-be-calibrated area is an area which is located within the range of the operation area of the mechanical arm and within the detection range of the visual sensor and is used for calibrating the mechanical arm;
step S104, controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor;
and step S106, determining the transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
The main body of the above steps may be a computing device located in the robot arm (in this case, there is a wired communication connection between the main body control part of the robot arm and the computing device), or may be a computing device independent from the outside of the robot arm, including a remote server, etc. (in this case, there is a wired or wireless communication connection between the main body control part of the robot arm and the computing device), but is not limited thereto.
The visual sensor applied in the method can be mounted on the mechanical arm, can also be an independent visual sensor, or can also be a visual sensor which is provided with the mechanical arm in a matching way and is independently mounted above the operation surface. The visual sensor may interact with a computing device on or separate from the robotic arm via a wired or wireless communication connection.
In step S104, the acquisition process of the second position information and the third position information has no requirement for the order from front to back.
Through the steps, the calibration path is generated based on the area to be calibrated, the plurality of calibration path points on the calibration path are selected, the mechanical arm is controlled to drive the calibration block to move according to the calibration path, the second position information of the calibration block under the base coordinate system of the mechanical arm and the third position information of the calibration block under the visual sensor coordinate system when the calibration block reaches each calibration path point are obtained in the moving process, and the transformation relation between the visual sensor coordinate system and the base coordinate system is determined according to the second position information and the third position information, so that the problem of low calibration efficiency in the hand-eye calibration process in the related technology can be solved, manual operation of the mechanical arm is not needed in the calibration process, the mechanical arm can automatically move according to the planned path and collect the position information, the operation process is simple, and the calibration efficiency is high.
In at least one exemplary embodiment, the step S102, the process of acquiring the first position information of the plurality of boundary points of the area to be calibrated may include: and controlling the mechanical arm to drive the calibration block to move to the plurality of boundary points of the area to be calibrated, and respectively collecting the flange positions when the mechanical arm drives the calibration block to move to the plurality of boundary points as the first position information.
In at least one exemplary embodiment, in step S102, generating a calibration path within the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path may include: determining a starting point and an end point in the boundary points, and generating a continuous calibration path without collision from the starting point to the end point in the range of the region to be calibrated; and selecting a plurality of calibration path points on the calibration path, and determining fourth position information of the plurality of calibration path points according to the first position information and the calibration path point selection interval. In the embodiment of the invention, the shape of the area to be calibrated can be any geometric shape, and when the calibration path is planned, uniform intervals are recommended to be adopted for planning as much as possible, and calibration path points are selected on the path according to the uniform intervals, so that the final calibration result is more accurate.
In at least one exemplary embodiment, the process of acquiring the second position information of the calibration block under the base coordinate system of the robot arm when the calibration block moves to each calibration path point in step S104 may include: when the mechanical arm drives the calibration block to move to each calibration path point, acquiring the flange position of the mechanical arm; and determining second position information of the calibration block under the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block relative to an origin of the flange coordinate system.
In at least one exemplary embodiment, determining the second position information of the calibration block under the base coordinate system according to the flange position and the coordinate system offset distance may include: for each calibration path point, the flange position corresponding to the current calibration path point is determined according to the following formula
And a coordinate system offset distance x0,y0,z0Determining second position information a of the calibration block in the base coordinate systemi(x,y,z)：
Wherein, c·Represents cos (. cndot.), s·Denotes sin (. cndot.), x0,y0,z0Are respectively as followsAn offset distance of an origin of a calibration block coordinate system relative to an origin of the flange coordinate system under x, y, and z axes, i ═ 1, 2.
In at least one exemplary embodiment, the process of acquiring, by the visual sensor, third position information of the calibration block in the visual sensor coordinate system when the calibration block moves to each calibration path point in step S104 may include: when the calibration block moves to each calibration path point, extracting the position of the central point of the calibration block on a preset plane through the visual sensor as third position information b of the calibration block in a visual sensor coordinate system when the calibration block moves to each calibration path pointi(x, y, z), i 1,2, k, k is the total number of the plurality of calibration path points.
In at least one exemplary embodiment, the process of determining the transformation relationship between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information in step S106 may include: according to R ═ VUTDetermining a rotation matrix R between the vision sensor coordinate system and the base coordinate system, and determining a translation vector t between the vision sensor coordinate system and the base coordinate system according to t-b-Ra,
wherein the content of the first and second substances,
airepresenting the second position information a of the calibration block under the base coordinate system when the calibration block moves to the ith calibration path pointi(x,y,z)，biThird position information b of the calibration block in the visual sensor coordinate system when the calibration block moves to the ith calibration path pointi(x, y, z), V and U are the pair matrices, respectively
According to H ═ U Λ VTA matrix obtained by matrix decomposition, T represents transposing the matrix, a'i＝ai-a，b’i＝bi-b, k are said plurality of calibration pathsThe total number of diameter points.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The embodiment further provides a calibration device for a mechanical arm, which is applied to a mechanical arm comprising a flange and a calibration block mounted at the tail end of the flange, and the device is used for implementing the above embodiments and preferred embodiments, and the description of the device is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 2 is a block diagram of a calibration apparatus for a robot arm according to an embodiment of the present invention, as shown in fig. 2, the apparatus includes:
the calibration path planning module 22 is configured to acquire first position information of a plurality of boundary points of an area to be calibrated, generate a calibration path within the range of the area to be calibrated according to the first position information, and determine a plurality of calibration path points on the calibration path;
the position calculation module 24 is configured to control the mechanical arm to drive the calibration block to move according to the calibration path, obtain second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and obtain third position information of the calibration block under a coordinate system of a visual sensor when the calibration block moves to each calibration path point through the visual sensor, where the region to be calibrated is a region located within a working region of the mechanical arm and within a detection range of the visual sensor and used for calibrating the mechanical arm;
a calibration result calculation module 26 configured to determine a transformation relationship between the visual sensor coordinate system and the base coordinate system based on the second position information and the third position information.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
The embodiment of the present invention further provides a calibration system for a mechanical arm, where the system is used to implement the foregoing embodiments and preferred embodiments, and details are not repeated for what has been described.
Fig. 3 is a schematic diagram of a calibration system of a robot arm according to an embodiment of the present invention, as shown in fig. 3, the calibration system of the robot arm includes: the robot comprises a mechanical arm 32, a calibration block 34, a vision sensor 36 and a computing device 36, wherein the mechanical arm 32 comprises a flange, the calibration block 34 is installed at the end of the flange, and wired or wireless communication connections are arranged between the mechanical arm 32 and the computing device 38 and between the vision sensor 36 and the computing device 38; the computing means 38 is arranged to:
acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path within the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path;
controlling the mechanical arm 32 to drive the calibration block 34 to move according to the calibration path, acquiring second position information of the calibration block 34 under a base coordinate system of the mechanical arm 32 when the calibration block 34 moves to each calibration path point, and acquiring third position information of the calibration block 34 under a visual sensor coordinate system when the calibration block 34 moves to each calibration path point through a visual sensor;
and determining a transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
The computing device 38 may be a computing device located in the robot arm 32 (in this case, there is a wired communication connection between the main control part of the robot arm 32 and the computing device), or may be a computing device independent from the outside of the robot arm 32, including a remote server, etc. (in this case, there is a wired or wireless communication connection between the main control part of the robot arm 32 and the computing device), but is not limited thereto. FIG. 3 shows only the external computing device 38, but those skilled in the art will appreciate that the computing device may also be mounted inside the robotic arm 32.
The vision sensor 36 in the above system may be mounted on the robot arm 32, may be a separate vision sensor, or may be a vision sensor provided in association with the robot arm 32 and separately mounted above the operation surface. The visual sensor 36 may interact with information via a wired or wireless communication connection with a computing device on the robotic arm 32 or provided separately from the robotic arm 32.
In at least one exemplary embodiment, the calculation device 38 is configured to obtain first position information of a plurality of boundary points of the area to be calibrated by: and controlling the mechanical arm 32 to drive the calibration block 34 to move to the plurality of boundary points of the area to be calibrated, and respectively collecting the flange positions when the mechanical arm 32 drives the calibration block 34 to move to the plurality of boundary points as the first position information.
In at least one exemplary embodiment, the computing device 38 is configured to obtain second position information of the calibration block 34 in the base coordinate system of the robotic arm 32 as the calibration block 34 moves to each calibration path point by: collecting the flange position of the mechanical arm 32 when the mechanical arm 32 drives the calibration block 34 to move to each calibration path point; and determining second position information of the calibration block 34 in the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block 34 relative to an origin of a flange coordinate system.
In at least one exemplary embodiment, the computing device 38 is configured to acquire, via the visual sensor 36, third positional information of the calibration block 34 in the visual sensor coordinate system as the calibration block 34 moves to each calibration waypoint by: extracting, by the vision sensor 36, a position of a center point of the calibration block 34 on a predetermined plane as the calibration block 34 moves to each calibration path point as third position information b of the calibration block 34 in a vision sensor coordinate system as the calibration block 34 moves to each calibration path pointi(x, y, z), i 1,2, k, k is the total number of the plurality of calibration path points.
Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above-mentioned method embodiments when executed.
In the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:
step S1, acquiring first position information of a plurality of boundary points of a to-be-calibrated area, generating a calibration path within the range of the to-be-calibrated area according to the first position information, and determining a plurality of calibration path points on the calibration path, wherein the to-be-calibrated area is an area which is located within the range of the operation area of the mechanical arm and within the detection range of the visual sensor and is used for calibrating the mechanical arm;
step S2, controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor;
step S3, determining a transformation relationship between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
In an exemplary embodiment, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
In an exemplary embodiment, the processor may be configured to execute the following steps by a computer program:
step S1, acquiring first position information of a plurality of boundary points of a to-be-calibrated area, generating a calibration path within the range of the to-be-calibrated area according to the first position information, and determining a plurality of calibration path points on the calibration path, wherein the to-be-calibrated area is an area which is located within the range of the operation area of the mechanical arm and within the detection range of the visual sensor and is used for calibrating the mechanical arm;
step S2, controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor;
step S3, determining a transformation relationship between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, and details of this embodiment are not repeated herein.
The following describes in detail a process of automatically calibrating a robot arm by taking calibration of the robot arm based on a rectangular region to be calibrated as an example. It should be noted that, in practical application, according to the application scene of the mechanical arm, various shapes, even three-dimensional regions to be calibrated can be set, and the scheme is also applicable.
The automatic hand-eye calibration method for the mechanical arm based on the 3D sensor in the embodiment is mainly realized based on the following four modules.
A first module: mechanical arm calibration path planning module
Before calibration is started, the pose of four corner points of a to-be-calibrated area (quadrangle) under a base coordinate system of a mechanical arm is obtained through the mechanical arm, the four points are input into the module, a path can be planned for the mechanical arm, fig. 4 is a schematic diagram of the calibration path in the to-be-calibrated area according to the embodiment of the invention, as shown in fig. 4, wherein black points represent path points of the mechanical arm, and the mechanical arm moves from a point a to a point b in a direction shown by an arrow in fig. 4 during automatic calibration. The determination scheme of the waypoints is as follows.
1) Judging according to the obtained four corner points
And
which two sides have a greater sum of their lengths, dividing the larger two sides into equal n equal parts, and dividing the smaller two sides into equal m equal parts, as shown in the following figure
The sides are longer, then each bisecting point is expressed as follows:
2) point djAnd point bm-jThe expression for the path point between two points is as follows:
therefore, the planning of the calibration path based on the four corner points is realized, and the position of each calibration path point is expressed based on the position coordinates of the four corner points.
And a second module: position calculation module of calibration block under mechanical arm base coordinate system
1) Firstly, adjusting the angle of a flange of the mechanical arm to 0 degree, then installing a calibration block on the flange, and obtaining a homogeneous transformation matrix from a coordinate system of the calibration block to a coordinate system of the flange of the mechanical arm according to the size and the installation mode of the calibration block as follows:
x0,y0,z0respectively, the offset distance of the origin of the coordinate system of the calibration block in the coordinate system of the flange relative to the origin of the coordinate system of the flange (this data can be measured after the flange is installed or can be calculated based on the size and installation mode of the calibration block).
2) The homogeneous transformation matrix from the flange coordinate system to the mechanical arm base coordinate system is as follows:
3) And calibrating a homogeneous transformation matrix from the block-based coordinate system to the mechanical arm-based coordinate system as follows:
the first three data of the fourth row of the matrix are the positions of the calibration block under the base coordinate system of the mechanical arm and are marked as ai(x,y,z),(i＝1,2,...,m*n)。
And a third module: position calculation module of calibration block under 3D camera coordinate system
The module extracts a central point of a designated plane of a calibration block according to point cloud information acquired by a 3D camera, wherein the point is the position of the calibration block in a 3D camera coordinate system and is marked as bi(x,y,z),(i＝1,2,...,m*n)。
And a module IV: hand-eye calibration result calculation module
1) The position of a calibration block obtained under the base coordinate system of the mechanical arm is recorded as A ═ a1,a2,...,am*n]And the position of the calibration block obtained in the camera coordinate system is recorded as B ═ B1,b2,...,bm*n]The same point of the calibration block is transformed under two coordinate systems into
bi＝Rai+t,(i＝1,2,...,m*n)
Where R is the rotation matrix and t is the translation vector.
2) Order to
a’i＝ai-a
b’i＝bi-b
H＝UΛVt
3) svd decompose the matrix H, rotate the matrix
R＝VUt
Translation vector
t＝b-Ra
FIG. 5 is a flow chart of a robot arm calibration scheme according to an embodiment of the present invention, as shown in FIG. 5, the robot arm calibration scheme includes the steps of:
step S501, a calibration block is installed at the tail end of a flange of a mechanical arm, the distance (x, y and z directions) from the central point of a to-be-identified surface of the calibration block to the original point of a flange coordinate system is measured, a camera is initialized, and the mechanical arm is started;
step S502, the software and the mechanical arm establish communication connection;
step S503, manually controlling the mechanical arm to move in sequence at four corner points of a quadrilateral space to be calibrated, ensuring that the distances between the four points and the camera are about dm (D is in the optimal visual field range of the 3D camera), and no obstacle exists in the space determined by the four points, and sequentially acquiring the position information of the mechanical arm flange at the four corner points by software;
step S504, software saves the position information of four points;
step S505, calling a calibration path planning module by software to obtain a collision-free calibration path;
step S506, the software sends the calibration path points to the mechanical arm to be executed;
step S507, the mechanical arm feeds back a signal to software after moving to the specified path point, and the software calls a base coordinate system position calculation module and a camera coordinate system position calculation module to obtain the positions of the same point of the fixed block in the two coordinate systems under the current scene and respectively stores the positions;
step S508, the software judges whether the route has already been all carried out, if not, repeat S506 value S508, otherwise continue step S509;
step S509, according to the point set information collected in step S507, the software calls the hand-eye calibration result calculation module to obtain a hand-eye calibration result.
It will be apparent to those skilled in the art that the various modules or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and they may be implemented using program code executable by the computing devices, such that they may be stored in a memory device and executed by the computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (11)
1. A calibration method of a mechanical arm is characterized by being applied to the mechanical arm comprising a flange and a calibration block arranged at the tail end of the flange, and the method comprises the following steps:
acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path within the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path;
controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor, wherein the acquiring of the second position information of the calibration block under the base coordinate system of the mechanical arm when the calibration block moves to each calibration path point comprises: when the mechanical arm drives the calibration block to move to each calibration path point, acquiring the flange position of the mechanical arm; determining second position information of the calibration block under the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block relative to an origin of the flange coordinate system;
and determining a transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
2. The method of claim 1, wherein obtaining first location information for a plurality of boundary points of an area to be calibrated comprises:
and controlling the mechanical arm to drive the calibration block to move to the plurality of boundary points of the area to be calibrated, and respectively collecting the flange positions when the mechanical arm drives the calibration block to move to the plurality of boundary points as the first position information.
3. The method of claim 1, wherein generating a calibration path within the region to be calibrated from the first position information, and determining a plurality of calibration path points on the calibration path comprises:
determining a starting point and an end point in the boundary points, and generating a continuous calibration path without collision from the starting point to the end point in the range of the region to be calibrated;
and selecting a plurality of calibration path points on the calibration path, and determining fourth position information of the plurality of calibration path points according to the first position information and the calibration path point selection interval.
4. The method of claim 1, wherein determining second position information of the calibration block in the base coordinate system based on the flange position and a coordinate system offset distance comprises:
for each calibration path point, the flange position corresponding to the current calibration path point is determined according to the following formula
And a coordinate system offset distance x0,y0,z0Determining second position information a of the calibration block in the base coordinate systemi(x,y,z)：
z＝-sθx0+cθsψy0+cθcψz0+y1
Wherein c represents cos(. s. represents sin (. x))0,y0,z0Offset distances of an origin of the calibration block coordinate system relative to an origin of the flange coordinate system under x, y and z axes are respectively, and i is 1, 2.
5. The method of claim 1, wherein obtaining, by the vision sensor, third positional information in the vision sensor coordinate system as the calibration block moves to each calibration waypoint comprises:
when the calibration block moves to each calibration path point, extracting the position of the central point of the calibration block on a preset plane through the visual sensor as third position information b of the calibration block in a visual sensor coordinate system when the calibration block moves to each calibration path pointi(x, y, z), i 1,2, k, k is the total number of the plurality of calibration path points.
6. The method of claim 1, wherein determining the transformation relationship between the vision sensor coordinate system and the base coordinate system based on the second location information and the third location information comprises:
according to R ═ VUTDetermining a rotation matrix R between the vision sensor coordinate system and the base coordinate system, and determining a translation vector t between the vision sensor coordinate system and the base coordinate system according to t-b-Ra,
wherein the content of the first and second substances,
airepresenting the second position information a of the calibration block under the base coordinate system when the calibration block moves to the ith calibration path pointi(x,y,z)，biThird position information b of the calibration block in the visual sensor coordinate system when the calibration block moves to the ith calibration path pointi(x, y, z), V and U are the pair matrices, respectively
According to H ═ U Λ VTA matrix obtained by matrix decomposition, T represents transposing the matrix, a'i＝ai-a，b′i＝bi-b, k is the total number of said plurality of nominal path points.
7. The utility model provides a calibration device of arm, its characterized in that is applied to the arm that includes the flange and install the calibration piece at the flange end, includes:
the calibration path planning module is used for acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path in the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path;
the position calculation module is configured to control the mechanical arm to drive the calibration block to move according to the calibration path, obtain second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and obtain third position information of the calibration block under a coordinate system of a visual sensor when the calibration block moves to each calibration path point through the visual sensor, wherein the position calculation module is configured to obtain the second position information of the calibration block under the base coordinate system of the mechanical arm when the calibration block moves to each calibration path point in the following manner: when the mechanical arm drives the calibration block to move to each calibration path point, acquiring the flange position of the mechanical arm; determining second position information of the calibration block under the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block relative to an origin of the flange coordinate system;
and the calibration result calculation module is set to determine the transformation relation between the visual sensor coordinate system and the base coordinate system according to the second position information and the third position information.
8. The calibration system of the mechanical arm is characterized by comprising the mechanical arm, a calibration block, a visual sensor and a computing device, wherein the mechanical arm comprises a flange, the calibration block is installed at the tail end of the flange, and wired or wireless communication connections are arranged between the mechanical arm and the computing device and between the visual sensor and the computing device;
the computing device is configured to:
acquiring first position information of a plurality of boundary points of an area to be calibrated, generating a calibration path within the range of the area to be calibrated according to the first position information, and determining a plurality of calibration path points on the calibration path;
controlling the mechanical arm to drive the calibration block to move according to the calibration path, acquiring second position information of the calibration block under a base coordinate system of the mechanical arm when the calibration block moves to each calibration path point, and acquiring third position information of the calibration block under a visual sensor coordinate system when the calibration block moves to each calibration path point through a visual sensor, wherein the computing device is configured to acquire the second position information of the calibration block under the base coordinate system of the mechanical arm when the calibration block moves to each calibration path point through the following modes: when the mechanical arm drives the calibration block to move to each calibration path point, acquiring the flange position of the mechanical arm; determining second position information of the calibration block under the base coordinate system according to the flange position and a coordinate system offset distance, wherein the coordinate system offset distance is an offset distance of an origin of a coordinate system of the calibration block relative to an origin of the flange coordinate system;
and determining a transformation relation between the vision sensor coordinate system and the base coordinate system according to the second position information and the third position information.
9. The system of claim 8, wherein the computing device is configured to obtain the first location information for the plurality of boundary points of the area to be calibrated by:
and controlling the mechanical arm to drive the calibration block to move to the plurality of boundary points of the area to be calibrated, and respectively collecting the flange positions when the mechanical arm drives the calibration block to move to the plurality of boundary points as the first position information.
10. The system of claim 8, wherein the computing device is configured to obtain, via the visual sensor, third positional information of the calibration block in the visual sensor coordinate system when the calibration block moves to each calibration waypoint by:
when the calibration block moves to each calibration path point, extracting the position of the central point of the calibration block on a preset plane through the visual sensor as third position information b of the calibration block in a visual sensor coordinate system when the calibration block moves to each calibration path pointi(x, y, z), i 1,2, k, k is the total number of the plurality of calibration path points.
11. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the method of any of claims 1 to 6 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010567739.6A CN111590593B (en) | 2020-06-19 | 2020-06-19 | Calibration method, device and system of mechanical arm and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010567739.6A CN111590593B (en) | 2020-06-19 | 2020-06-19 | Calibration method, device and system of mechanical arm and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111590593A CN111590593A (en) | 2020-08-28 |
| CN111590593B true CN111590593B (en) | 2021-12-17 |
Family
ID=72188106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010567739.6A Active CN111590593B (en) | 2020-06-19 | 2020-06-19 | Calibration method, device and system of mechanical arm and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111590593B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112932670B (en) * | 2020-11-07 | 2022-02-08 | 北京和华瑞博医疗科技有限公司 | Calibration method, mechanical arm control method and surgical operation system |
| CN113269836A (en) * | 2021-03-30 | 2021-08-17 | 深圳市世宗自动化设备有限公司 | 3D camera calibration method and device, computer equipment and storage medium thereof |
| CN113894800B (en) * | 2021-09-27 | 2023-08-11 | 深圳市启玄科技有限公司 | Multi-point teaching-based regular grid welding method, device, terminal and storage medium |
| CN114092335B (en) * | 2021-11-30 | 2023-03-10 | 群滨智造科技(苏州)有限公司 | Image splicing method, device and equipment based on robot calibration and storage medium |
| CN114920000A (en) * | 2022-04-22 | 2022-08-19 | 歌尔科技有限公司 | Conveying device, mechanical equipment and control method of conveying device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9639935B1 (en) * | 2016-05-25 | 2017-05-02 | Gopro, Inc. | Apparatus and methods for camera alignment model calibration |
| CN109732601A (en) * | 2019-01-04 | 2019-05-10 | 上海交通大学 | A kind of automatic Calibration robot pose method and apparatus vertical with camera optical axis |
| CN109807885A (en) * | 2018-12-29 | 2019-05-28 | 深圳市越疆科技有限公司 | A kind of vision calibration method of manipulator, device and intelligent terminal |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5164811B2 (en) * | 2008-11-26 | 2013-03-21 | キヤノン株式会社 | Work system and information processing method |
| CN103279949B (en) * | 2013-05-09 | 2015-10-07 | 浙江大学 | Based on the multi-camera parameter automatic calibration system operation method of self-align robot |
| WO2016040229A1 (en) * | 2014-09-10 | 2016-03-17 | Faro Technologies, Inc. | Method for optically measuring three-dimensional coordinates and calibration of a three-dimensional measuring device |
| KR102576842B1 (en) * | 2017-01-04 | 2023-09-12 | 삼성전자주식회사 | Robot and electronic device for performing hand-eye calibration |
| CN109702738B (en) * | 2018-11-06 | 2021-12-07 | 深圳大学 | Mechanical arm hand-eye calibration method and device based on three-dimensional object recognition |
| CN110293559B (en) * | 2019-05-30 | 2023-03-28 | 上海理工大学 | Installation method for automatically identifying, positioning and aligning |
| CN110244717B (en) * | 2019-06-03 | 2020-08-07 | 武汉理工大学 | Port crane climbing robot automatic path finding method based on existing three-dimensional model |
| CN110695982A (en) * | 2019-10-17 | 2020-01-17 | 南京隆越自动化科技有限公司 | Mechanical arm hand-eye calibration method and device based on three-dimensional vision |
| CN110861076A (en) * | 2019-12-11 | 2020-03-06 | 深圳市盛世鸿恩科技有限公司 | Hand eye calibration device of mechanical arm |
| CN110977987B (en) * | 2019-12-25 | 2021-07-20 | 杭州未名信科科技有限公司 | Mechanical arm hand-eye calibration method, device and system |
-
2020
- 2020-06-19 CN CN202010567739.6A patent/CN111590593B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9639935B1 (en) * | 2016-05-25 | 2017-05-02 | Gopro, Inc. | Apparatus and methods for camera alignment model calibration |
| CN109807885A (en) * | 2018-12-29 | 2019-05-28 | 深圳市越疆科技有限公司 | A kind of vision calibration method of manipulator, device and intelligent terminal |
| CN109732601A (en) * | 2019-01-04 | 2019-05-10 | 上海交通大学 | A kind of automatic Calibration robot pose method and apparatus vertical with camera optical axis |
Non-Patent Citations (2)
| Title |
|---|
| Isela Bonilla ; Marco Mendoza ; Emilio J. Gonzalez-Galván.Path-Tracking Maneuvers With Industrial Robot Manipulators Using Uncalibrated Vision and Impedance Control.《Systems, Man, and Cybernetics, Part C (Applications and Reviews)》.2012, * |
| 基于视觉的机械臂控制技术研究;刘晓坤;《中国优秀硕士学位论文全文数据库 (信息科技辑)》;20140415(第4期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111590593A (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111590593B (en) | Calibration method, device and system of mechanical arm and storage medium | |
| US11911914B2 (en) | System and method for automatic hand-eye calibration of vision system for robot motion | |
| US10585167B2 (en) | Relative object localization process for local positioning system | |
| EP1555508B1 (en) | Measuring system | |
| EP1584426B1 (en) | Tool center point calibration system | |
| US20200298411A1 (en) | Method for the orientation of an industrial robot, and industrial robot | |
| US6816755B2 (en) | Method and apparatus for single camera 3D vision guided robotics | |
| CN111025283B (en) | Method and device for linking radar and dome camera | |
| US20050273199A1 (en) | Robot system | |
| US20160039094A1 (en) | Robot System And Method For Calibration | |
| KR20170017786A (en) | System and method for tying together machine vision coordinate spaces in a guided assembly environment | |
| US11090810B2 (en) | Robot system | |
| JP2004508954A (en) | Positioning device and system | |
| JPH0435885A (en) | Calibration method for visual sensor | |
| CN110815205A (en) | Calibration method, system and device of mobile robot | |
| CN114310901B (en) | Coordinate system calibration method, device, system and medium for robot | |
| WO2018043524A1 (en) | Robot system, robot system control device, and robot system control method | |
| CN111982012A (en) | Three-dimensional measurement method, device and system for component surface | |
| KR101782317B1 (en) | Robot calibration apparatus using three-dimensional scanner and robot calibration method using the same | |
| CN115284292A (en) | Mechanical arm hand-eye calibration method and device based on laser camera | |
| CN115439633A (en) | Calibration method and device and electronic equipment | |
| CN112743548B (en) | Method, system and terminal for unifying hand-eye calibration of two mechanical arms | |
| KR20180017503A (en) | Method and System for Calibration of Mobile Robot and Camera | |
| JP7278637B2 (en) | Self-propelled moving device | |
| CN116563491B (en) | Digital twin scene modeling and calibration method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |