CN109215086A - Camera extrinsic scaling method, equipment and system - Google Patents
Camera extrinsic scaling method, equipment and system Download PDFInfo
- Publication number
- CN109215086A CN109215086A CN201810979181.5A CN201810979181A CN109215086A CN 109215086 A CN109215086 A CN 109215086A CN 201810979181 A CN201810979181 A CN 201810979181A CN 109215086 A CN109215086 A CN 109215086A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- camera
- marker
- coordinates
- plane
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000003550 marker Substances 0.000 claims abstract description 112
- 239000011159 matrix material Substances 0.000 claims abstract description 51
- 238000013519 translation Methods 0.000 claims abstract description 22
- 239000013598 vector Substances 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000036544 posture Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The present invention provides a kind of Camera extrinsic scaling method, equipment and system, it is related to technical field of robot vision, first the marker that repeatedly shoots of acquisition camera formed uncalibrated image in the plane mirror under different angle；Then coordinate of the actual tag point in marker coordinate system in marker is transformed into camera coordinates system, establish the first equation indicated by spin matrix and transposed matrix, according to the intersection of plane where the plane mirror under two different angles, and the positional relationship between corresponding mirror surface index point, establish second equation；Coordinate by mirror surface index point in the relative positional relationship under camera coordinates system and the index point in uncalibrated image in pixel coordinate system obtains coordinate of the mirror surface index point in camera coordinates system；By the first equation of simultaneous and second equation, and the coordinate according to mirror surface index point in camera coordinates system, acquire spin matrix and translation matrix.This can effectively improve the Efficiency and accuracy of Camera extrinsic calibration.
Description
Technical Field
The invention relates to the technical field of robot vision, in particular to a camera external reference calibration method, equipment and a system.
Background
In the camera external reference calibration method in the prior art, a marker is usually used to calibrate a rotation matrix and a translation matrix of a camera coordinate system relative to a coordinate system on a selected reference object, the marker needs to be shot for many times to obtain marker images of multiple angles, and the images are used to calibrate the camera external reference. However, this method can only be implemented on the premise that the camera can see the marker, and when the camera cannot see the marker, it is obvious that the external reference matrix of the camera coordinate system relative to the coordinate system on the marker cannot be directly calibrated; the solution widely adopted at present is to make the camera shoot the image formed by the marker in the plane mirror by means of the plane mirror, move the plane mirror to a plurality of different positions and postures (poses), and calculate the external parameter matrix of the camera by means of shooting again.
At present, on the problem of realizing external reference calibration of a camera by means of plane mirrors, whether the number of the plane mirrors and the position and posture of the plane mirrors relative to the camera need to be determined in advance mainly presents two different methods: one is that 5 plane mirrors are placed under different poses, and the poses of the plane mirrors relative to the camera do not need to be determined in advance; and the other is that 3 plane mirrors are placed under different poses, and the poses of the plane mirrors relative to the camera need to be determined in advance. In the former method, the number of plane mirrors required to be placed is large, and the workload corresponding to the plane mirrors is also large; in the latter method, the position of the plane mirror relative to the camera is difficult to accurately measure because the origin of the camera coordinate system, i.e., the optical center position, is unknown.
Disclosure of Invention
In view of this, the present invention provides a method, a device and a system for calibrating external parameters of a camera, which can effectively improve the efficiency and accuracy of calibrating the external parameters of the camera.
In a first aspect, an embodiment of the present invention provides a method for calibrating external parameters of a camera, where the method includes:
when the plane mirror rotates to different angles for multiple times, acquiring calibration images shot by the camera for multiple times; the calibration image is an image of the marker in the plane mirror;
converting the coordinates of the actual mark points in the marker in a coordinate system of the marker into a coordinate system of a camera, and establishing a first equation represented by a rotation matrix and a translation matrix;
establishing a second equation according to the position relationship between the intersection line of the planes where the plane mirrors are positioned at two different angles and the corresponding mirror surface mark points; the mirror surface mark point is an image of a corresponding actual mark point in the marker in the plane mirror;
determining the coordinates of the mirror surface mark points in the camera coordinate system according to the coordinates of the mark points in the calibration image in the pixel coordinate system;
and simultaneously establishing the first equation and the second equation, and solving the rotation matrix and the translation matrix according to the coordinates of the mirror surface mark points in a camera coordinate system.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the marker includes at least 4 preset actual marker points, and the marker coordinate system is established with a center of a pattern formed by the actual marker points as an origin.
With reference to the first possible implementation manner of the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the converting coordinates of the actual marker point in the marker coordinate system to the camera coordinate system and establishing a first equation represented by a rotation matrix and a translation matrix is as follows:
pi＝R·Lpi+T
wherein i is the ith actual marker point in the marker, and i is 1,2, … …, N; p is a radical ofiThe coordinates of the actual mark point in the camera coordinate system,Lpithe coordinate of the actual mark point in the marker coordinate system is shown, R is a rotation matrix converted from the marker coordinate system where the actual mark point is located to a camera coordinate system, and T is a translation matrix converted from the marker coordinate system where the actual mark point is located to the camera coordinate system.
With reference to the second possible implementation manner of the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where after the steps of converting the coordinates of the actual marker point in the marker coordinate system to the camera coordinate system, and establishing the first equation represented by the rotation matrix and the translation matrix, the method further includes:
and (2) the coordinates of the actual mark points in the marker and the corresponding mirror mark points in the camera coordinate system and the position relation between the plane mirror and the camera are as follows:
substituting the first equation to obtain:
wherein,the coordinates of the mirror marking point in the camera coordinate system,is the vertical distance from the ith actual mark point to the plane mirror, dmIs the vertical distance, n, from the plane mirror to the origin of the camera coordinate systemmIs the normal vector of the plane where the plane mirror is located.
With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the step of establishing a second equation according to a position relationship between an intersection line of planes of the plane mirrors at two different angles and corresponding mirror surface mark points in the plane mirrors at the two different angles includes:
the intersection line of the planes of the plane mirrors at two different angles is perpendicular to the connecting line of the corresponding mirror surface mark points in the plane mirrors at the two different angles, and the second equation is established as follows:
wherein,andthe mirror surface mark points are formed in the plane mirrors under two different angles by the same actual mark point;
Cmm'is the intersection line of the planes of the plane mirrors under the two different angles:
Cmm'＝nm×nm'
wherein n ismAnd nm'Are respectively normal vectors of the plane where the plane mirrors under two different angles are located.
With reference to the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein the pixel coordinate system is established with a center of the calibration image as an origin.
With reference to the first or fifth possible implementation manner of the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where the step of obtaining the coordinates of the mirror surface mark point in the camera coordinate system according to the relative position relationship of the mirror surface mark point in the camera coordinate system and the coordinates of the mark point in the calibration image in the pixel coordinate system includes:
obtaining the relative position relation of the mirror surface mark point in the camera coordinate system according to the preset relative position relation of the actual mark point in the marker coordinate system;
obtaining the coordinates of the mark points in the calibration image in the pixel coordinate system according to the established pixel coordinate system;
and obtaining the coordinates of the mirror surface mark points in the camera coordinate system through a PnP algorithm according to the relative position relation of the mirror surface mark points in the camera coordinate system and the coordinates of the mark points in the calibration image in the pixel coordinate system.
In a second aspect, an embodiment of the present invention further provides a camera external reference calibration device, where the device includes a memory, a processor, and a data interface;
the memory is stored with a computer program operable on the processor, the processor implementing the steps of the method of any one of the first aspect when executing the computer program;
and the data interface is used for transmitting the calibration image to the processor, and the processor processes the calibration image.
In a third aspect, an embodiment of the present invention further provides a camera external reference calibration system, where the system includes: the plane mirror, the marker, the robot, the camera, the rotating table and the camera external reference calibration device in the second aspect;
the marker and the camera are mounted on the robot; the plane mirror is arranged on the rotating platform; the rotating platform is used for rotating the plane mirror for multiple times to different angles so that the camera can shoot calibration images formed by the marker in the plane mirror for multiple times and calibrate the external parameters of the camera.
With reference to the third aspect, an embodiment of the present invention provides a first possible implementation manner of the third aspect, wherein the rotating table includes at least 3 preset rotation angles.
The embodiment of the invention has the following beneficial effects:
the method, the device and the system for calibrating the external parameters of the camera provided by the embodiment of the invention firstly collect calibration images formed by a marker shot by the camera in plane mirrors at different angles for many times; then converting the coordinate of the actual mark point in the marker in a coordinate system of the marker into a coordinate system of a camera, establishing a first equation represented by a rotation matrix and a translation matrix, and establishing a second equation according to the intersection line of the planes of the plane mirrors under two different angles and the position relation between the corresponding mirror mark points in the plane mirrors; obtaining the coordinates of the mirror surface mark points in a camera coordinate system according to the relative position relation of the mirror surface mark points in the camera coordinate system and the coordinates of the mark points in the calibration image in a pixel coordinate system; and solving a rotation matrix and a translation matrix by combining a first equation and a second equation and according to the coordinates of the mirror surface mark points in the camera coordinate system. The efficiency and the accuracy of external parameter calibration of the camera can be effectively improved.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part may be learned by the practice of the above-described techniques of the disclosure, or may be learned by practice of the disclosure.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a camera external reference calibration method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a camera external reference calibration method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a camera external reference calibration method according to an embodiment of the present invention;
fig. 4 is a block diagram of a camera external reference calibration device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a camera external reference calibration system according to an embodiment of the present invention.
Icon:
1-a robot; 2-a camera; 3-a marker; 4-rotating table; 5-plane mirror.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, on the problem of realizing external reference calibration of a camera by means of plane mirrors, whether the number of the plane mirrors and the position and posture of the plane mirrors relative to the camera need to be determined in advance mainly presents two different methods: one is that 5 plane mirrors are placed under different poses, and the poses of the plane mirrors relative to the camera do not need to be determined in advance; and the other is that 3 plane mirrors are placed under different poses, and the poses of the plane mirrors relative to the camera need to be determined in advance. In the former method, the number of plane mirrors required to be placed is large, and the workload corresponding to the plane mirrors is also large; in the latter method, the position of the plane mirror relative to the camera is difficult to accurately measure because the origin of the camera coordinate system, i.e., the optical center position, is unknown. Based on the above, the method, the device and the system for calibrating the external reference of the camera provided by the embodiment of the invention are simple to operate, and can effectively improve the efficiency and the accuracy of calibrating the external reference of the camera.
To facilitate understanding of the embodiment, a detailed description is first given to a camera external reference calibration method disclosed in the embodiment of the present invention.
Example one
The present embodiment provides a camera referencing method, it should be noted that the steps shown in the flowchart of the figure may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different than here. The present embodiment will be described in detail below.
Fig. 1 shows a flowchart of a camera external reference calibration method according to an embodiment of the present invention, and as shown in fig. 1, the method includes the following steps:
s100, when the plane mirror rotates for multiple times to different angles, acquiring calibration images shot by a camera for multiple times; the calibration image is an image of the marker in the plane mirror.
The plane mirror rotates to 3 different angles at least, and the camera shoots an image formed by the marker in the plane mirror as a calibration image when the plane mirror rotates to the different angles once, wherein the image at least comprises 3 calibration images.
S102, converting the coordinates of the actual mark points in the marker in the coordinate system of the marker into a coordinate system of a camera, and establishing a first equation represented by a rotation matrix and a translation matrix.
The marker comprises at least 4 preset actual marker points, and a marker coordinate system is established by taking the center of a pattern formed by the actual marker points as an origin, so that the coordinates of the actual marker points in the marker coordinate system and the mutual position relation of the actual marker points in the marker coordinate system can be obtained.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a method for calibrating external parameters of a camera according to an embodiment of the present invention. Alternatively, let piThe coordinates of the actual landmark points in the camera coordinate system,Lpifor the coordinates of the actual marker point in the marker coordinate system, the coordinates of the actual marker point in the marker coordinate system are converted to the camera coordinate system and a first equation represented by a rotation matrix and a translation matrix is established:
pi＝R·Lpi+T (1)
wherein, i is the ith actual mark point in the marker, i is 1,2, … …, N; r is a rotation matrix converted from the marker coordinate system where the actual marker points are located to the camera coordinate system, and T is a translation matrix converted from the marker coordinate system where the actual marker points are located to the camera coordinate system.
In addition, assume that the plane of the plane mirror is pimThe image of the actual mark point in the plane mirror is the mirror mark point;the coordinates of the i-th mirror index point in the camera coordinate system,is the vertical distance from the ith actual mark point to the plane mirror, dmIs the vertical distance of the plane mirror from the origin of the camera coordinate system, nmIs a plane pi of the plane mirrormNormal vector of (1), thenFor the coordinates of the i-th mirror surface index point in the camera coordinate system along the normal vector nmIs projected.
The coordinates of the actual mark points and the corresponding mirror mark points in the marker in the camera coordinate system have the following relationship with the plane mirror and the camera:
substituting this relationship into the first equation yields:
and S104, establishing a second equation according to the position relation between the intersection line of the planes of the plane mirrors at two different angles and the corresponding mirror surface mark points.
Referring to fig. 3, fig. 3 is a schematic diagram of another camera external reference calibration method according to an embodiment of the present invention. Assuming that the plane pi of the plane mirror under two different angles ismAnd pim'The intersection line of is Cmm'According to the properties of the two intersecting planes, the intersection line can be obtained to be perpendicular to the connecting line of the corresponding mirror surface mark points in the plane mirrors under two different angles, and thus a second equation is established as follows:
wherein i is 1,2, … …, N;andthe coordinates of mirror surface mark points formed by the ith actual mark point in the plane mirrors under two different angles in a camera coordinate system are respectively;
Cmm'＝nm×nm'(5)
wherein n ismAnd nm'Plane pi of plane mirror under two different anglesmAnd pim'The normal vector of (2).
S106, determining the coordinates of the mirror surface mark points in the camera coordinate system according to the coordinates of the mark points in the calibration image in the pixel coordinate system.
The pixel coordinate system is established by taking the center of the calibration image as an origin, so that the coordinates of the mark point in the calibration image in the pixel coordinate system can be obtained.
And obtaining the coordinates of the mirror surface mark point in the camera coordinate system through a PnP algorithm according to the preset relative position relation of the actual mark point in the marker coordinate system and the coordinates of the mark point in the calibration image in the pixel coordinate system. Optionally, the PnP algorithm may be implemented by a SolvePNP function in OpenCV.
And S108, combining the first equation and the second equation, and solving a rotation matrix and a translation matrix according to the coordinates of the mirror surface mark points in the camera coordinate system.
C can be calculated according to the second equation and the obtained coordinates of the mark points of the mirror surface in the camera coordinate systemmm'Furthermore, because the intersection line of the planes of the plane mirrors under two different angles is Cmm'Perpendicular to the normal vector of the plane where the plane mirrors under two different angles are located, there are:
thereby obtaining a normal vector nm。
When the plane mirrors rotate to K different angles, the plane mirrors under every two adjacent angles form K-1 intersecting lines, K-1 equations in the form of the equation (6) are totally formed, and the K-1 equations are combined into:
Sj·nj＝0 (7)
wherein S isj＝(Cj1Cj2...Cj(j-1)Cjj...Cj(k-1))T，SjIs a matrix of (K-1) × 3.
Further, the normal vector n of the plane where the plane mirror is located under each angle is obtainedj。
When there are N actual mark points and plane mirrors rotating by k different angles, the coordinates of the calculated mirror mark points in the camera coordinate systemAnd the normal vector n of the plane where the plane mirror is located under each anglekAnd the coordinates of each actual marker point in the marker coordinate systemLpiSubstituting equation (3) for the following linear equation set, which can be abbreviated as:
AZ＝B (8)
wherein,
wherein r is1、r2And r3The first, second and third columns of the rotation matrix R.
Therefore, the formula (8) has 12+ k position parameters and 3 × N × k equations, and when k is greater than or equal to 3 and 3 × N × k is greater than or equal to 12+ k, the formula (8) is solved to obtain a plurality of sets of solutions meeting the conditions.
And then, carrying out re-projection on the multiple groups of solutions to obtain a solution with the minimum error as a solution meeting the condition. Therefore, a rotation matrix and a translation matrix are obtained, and external reference calibration of the camera is realized. Optionally, the reprojection of the multiple sets of solutions may be implemented by a ProjectPoints function in OpenCV.
The camera external reference calibration method provided by the embodiment of the invention comprises the steps of firstly collecting calibration images formed by a marker which is shot by a camera for many times in plane mirrors under different angles; then converting the coordinate of the actual mark point in the marker in a coordinate system of the marker into a coordinate system of a camera, establishing a first equation represented by a rotation matrix and a translation matrix, and establishing a second equation according to the intersection line of the planes of the plane mirrors under two different angles and the position relation between the corresponding mirror mark points in the plane mirrors; obtaining the coordinates of the mirror surface mark points in a camera coordinate system according to the relative position relation of the mirror surface mark points in the camera coordinate system and the coordinates of the mark points in the calibration image in a pixel coordinate system; and solving a rotation matrix and a translation matrix by combining a first equation and a second equation and according to the coordinates of the mirror surface mark points in the camera coordinate system. The efficiency and the accuracy of external parameter calibration of the camera can be effectively improved.
Example two
On the basis of the above embodiments, the embodiment of the present disclosure further provides a camera external reference calibration apparatus, as shown in fig. 4, the camera external reference calibration apparatus includes a processor 110, a memory 120 connected to the processor 110, a data interface 130, and the like.
The memory 120 may be used for storing software programs, such as program instructions corresponding to the camera external reference calibration method in the embodiment of the present invention, and the processor 110 executes various corresponding functional applications and data processing by executing the software programs stored in the memory 120. The memory 120 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the control system (such as the established first equation and second equation), and the like. Further, the memory 120 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The processor 110 is a control center of the camera external reference calibration apparatus, connects various parts of the entire apparatus using various interfaces and lines, and performs various functions and processes data by running or executing software programs and/or modules stored in the memory 120 and calling data stored in the memory 120, thereby performing overall monitoring of the camera external reference calibration system. Optionally, processor 110 may include one or more processing units.
The data interface 130 may include a USB interface, a serial interface, an infrared interface or a bluetooth interface, and is used for transmitting the calibration image to the processor 110, and the processor 110 performs data processing on the calibration image.
Further, the embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions, and when the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the above-mentioned camera external reference calibration method.
EXAMPLE III
On the basis of the above embodiments, the embodiments of the present disclosure further provide a camera external reference calibration system, as shown in fig. 5, the system includes: the robot 1, the camera 2, the marker 3, the rotating platform 4, the plane mirror 5 and the camera external reference calibration device provided by the embodiment;
the marker 3 and the camera 2 are mounted on the robot 1; the plane mirror 5 is arranged on the rotating platform 4; the rotating platform 4 is used for rotating the plane mirror 5 to different angles for multiple times, so that the camera 2 captures calibration images of the marker in the plane mirror 5 for multiple times.
Optionally, the camera external parameter calibration device may be disposed inside the camera, and is configured to calibrate the camera external parameter.
The rotating table comprises at least three preset rotating angles, so that the plane mirror 5 can rotate to at least three different angles.
The robot 1 comprises a base and a body connected to each other. The marker 3 is fixedly arranged on the base; the camera 2 is a 2D camera, is fixedly arranged in the middle of the outer wall of the machine body and is used for shooting a calibration image formed by the marker 3 in the plane mirror 5;
alternatively, the camera 2 and the marker 3 may be fixedly installed on the same side of the robot 1, so that the camera 2 can completely capture a calibration image of the marker 3 in the plane mirror 5.
The marker 3 comprises at least four preset marker points which are positioned on the same plane of the marker 3.
Alternatively, the marker 3 may be rectangular, and the individual marker points on the marker 3 may constitute a simple geometric figure. Optionally, the marker 3 includes four marker points, the marker points may be circular, and the four marker points form a rectangle, so as to establish a coordinate system with a center of the rectangle formed by the marker points as an origin center, and obtain coordinates of a circle center of each marker point in its own coordinate system.
Optionally, the marker 3 may also be a black-and-white checkerboard, the marker points are corner points at the intersection of the black-and-white rectangles, a marker coordinate system may be established with the center of the black-and-white checkerboard as an origin, and coordinates of each corner point in the marker coordinate system are obtained.
The method comprises the steps of establishing a robot coordinate system by taking the center of a robot base as an origin, determining the position relation between the robot coordinate system and the marker coordinate system in advance when the marker is installed, solving a rotation matrix and a translation matrix of the marker coordinate system relative to the camera coordinate system according to the camera external reference calibration method provided by the embodiment, and obtaining a transformation matrix of the camera coordinate system relative to the robot coordinate system, so that the camera coordinate system is converted into the robot coordinate system, and the accurate navigation of the robot can be realized.
The camera external reference calibration system provided by the embodiment of the invention has the same technical characteristics as the camera external reference calibration method provided by the embodiment, so that the same technical problems can be solved, and the same technical effects can be achieved.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "inside", and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, which are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and operate, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A camera external reference calibration method is characterized by comprising the following steps:
when the plane mirror rotates to different angles for multiple times, acquiring calibration images shot by the camera for multiple times; the calibration image is an image of the marker in the plane mirror;
converting the coordinates of the actual mark points in the marker in a coordinate system of the marker into a coordinate system of a camera, and establishing a first equation represented by a rotation matrix and a translation matrix;
establishing a second equation according to the position relationship between the intersection line of the planes where the plane mirrors are positioned at two different angles and the corresponding mirror surface mark points; the mirror surface mark point is an image of a corresponding actual mark point in the marker in the plane mirror;
determining the coordinates of the mirror surface mark points in the camera coordinate system according to the coordinates of the mark points in the calibration image in the pixel coordinate system;
and simultaneously establishing the first equation and the second equation, and solving the rotation matrix and the translation matrix according to the coordinates of the mirror surface mark points in a camera coordinate system.
2. The method of claim 1, wherein the marker includes at least 4 actual marker points set in advance, and the marker coordinate system is established with the center of the pattern composed of the actual marker points as an origin.
3. The method of claim 2, wherein the converting coordinates of actual marker points in the marker coordinate system to the camera coordinate system and establishing a first equation represented by a rotation matrix and a translation matrix is:
pi＝R·Lpi+T
wherein i is the ith actual marker point in the marker, and i is 1,2, … …, N; p is a radical ofiThe coordinates of the actual mark point in the camera coordinate system,Lpithe coordinate of the actual mark point in the marker coordinate system is shown, R is a rotation matrix converted from the marker coordinate system where the actual mark point is located to a camera coordinate system, and T is a translation matrix converted from the marker coordinate system where the actual mark point is located to the camera coordinate system.
4. The method of claim 3, wherein after the steps of converting the coordinates of the actual marker point in the marker coordinate system to the camera coordinate system and establishing the first equation represented by the rotation matrix and the translation matrix, the method further comprises:
and (2) the coordinates of the actual mark points in the marker and the corresponding mirror mark points in the camera coordinate system and the position relation between the plane mirror and the camera are as follows:
substituting the first equation to obtain:
wherein,the coordinates of the mirror marking point in the camera coordinate system,is the vertical distance from the ith actual mark point to the plane mirror, dmIs the vertical distance, n, from the plane mirror to the origin of the camera coordinate systemmIs the normal vector of the plane where the plane mirror is located.
5. The method of claim 1, wherein the step of establishing a second equation based on the position relationship between the intersection of the planes of the two flat mirrors at two different angles and the corresponding mirror mark points of the two flat mirrors at the two different angles comprises:
the intersection line of the planes of the plane mirrors at two different angles is perpendicular to the connecting line of the corresponding mirror surface mark points in the plane mirrors at the two different angles, and the second equation is established as follows:
wherein,andthe mirror surface mark points are formed in the plane mirrors under two different angles by the same actual mark point;
Cmm'is the intersection line of the planes of the plane mirrors under the two different angles:
Cmm'＝nm×nm'
wherein n ismAnd nm'Are respectively normal vectors of the plane where the plane mirrors under two different angles are located.
6. The method of claim 1, wherein the pixel coordinate system is established with the center of the calibration image as an origin.
7. The method according to claim 2 or 6, wherein the step of obtaining the coordinates of the mirror surface mark point in the camera coordinate system according to the coordinates of the mark point in the calibration image in the pixel coordinate system comprises:
obtaining the coordinates of the mark points in the calibration image in the pixel coordinate system according to the established pixel coordinate system;
and obtaining the coordinates of the mirror surface mark point in the camera coordinate system through a PnP algorithm according to the preset relative position relation of the actual mark point in the marker coordinate system and the coordinates of the mark point in the calibration image in the pixel coordinate system.
8. A camera external parameter calibration device comprises a memory, a processor and a data interface;
a computer program that can be run on the processor is stored in the memory, and the steps of the method according to any one of claims 1 to 7 are implemented when the computer program is executed by the processor;
and the data interface is used for transmitting the calibration image to the processor, and the processor processes the calibration image.
9. A camera external reference calibration system, the system comprising: a plane mirror, a marker, a robot, a camera, a rotating stage, and the camera external reference calibration apparatus described in claim 8;
the marker and the camera are mounted on the robot; the plane mirror is arranged on the rotating platform; the rotating platform is used for rotating the plane mirror for multiple times to different angles so that the camera can shoot calibration images formed by the marker in the plane mirror for multiple times and calibrate the external parameters of the camera.
10. The camera external reference calibration system is characterized in that the rotating table comprises at least 3 preset rotating angles.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810979181.5A CN109215086A (en) | 2018-08-24 | 2018-08-24 | Camera extrinsic scaling method, equipment and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810979181.5A CN109215086A (en) | 2018-08-24 | 2018-08-24 | Camera extrinsic scaling method, equipment and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109215086A true CN109215086A (en) | 2019-01-15 |
Family
ID=64989667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810979181.5A Pending CN109215086A (en) | 2018-08-24 | 2018-08-24 | Camera extrinsic scaling method, equipment and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109215086A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109949374A (en) * | 2019-04-26 | 2019-06-28 | 清华大学深圳研究生院 | A kind of reversed camera calibration system and method based on mirror image |
| CN110370316A (en) * | 2019-06-20 | 2019-10-25 | 重庆大学 | It is a kind of based on the robot TCP scaling method vertically reflected |
| CN111652945A (en) * | 2020-06-03 | 2020-09-11 | 北京方程奇迹科技有限公司 | Camera calibration method |
| CN112215884A (en) * | 2019-07-09 | 2021-01-12 | 北京地平线机器人技术研发有限公司 | Pose determination method and device for plane marker |
| CN112654886A (en) * | 2020-05-27 | 2021-04-13 | 华为技术有限公司 | External parameter calibration method, device, equipment and storage medium |
| CN112975959A (en) * | 2021-02-10 | 2021-06-18 | 山东英信计算机技术有限公司 | Machine vision-based radiator assembling and positioning method, system and medium |
| CN113592951A (en) * | 2021-07-14 | 2021-11-02 | 阿波罗智联(北京)科技有限公司 | Method and device for calibrating external parameters of vehicle-road cooperative middle-road side camera and electronic equipment |
| CN116883517A (en) * | 2023-09-07 | 2023-10-13 | 华东交通大学 | Camera parameter calibration method without overlapping view fields based on plane mirror |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106157322A (en) * | 2016-07-28 | 2016-11-23 | 北京航空航天大学 | A kind of camera installation site scaling method based on plane mirror |
-
2018
- 2018-08-24 CN CN201810979181.5A patent/CN109215086A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106157322A (en) * | 2016-07-28 | 2016-11-23 | 北京航空航天大学 | A kind of camera installation site scaling method based on plane mirror |
Non-Patent Citations (3)
| Title |
|---|
| KOSUKE TAKAHASHI等: "A New Mirror-based Extrinsic Camera Calibration Using an Orthogonality Constraint", 《2012 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 张广军等: "移动机器人足目标定方法", 《机器人》 * |
| 张旭等: "基于镜面反射的手眼标定方法", 《中国机械工程》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109949374A (en) * | 2019-04-26 | 2019-06-28 | 清华大学深圳研究生院 | A kind of reversed camera calibration system and method based on mirror image |
| CN110370316A (en) * | 2019-06-20 | 2019-10-25 | 重庆大学 | It is a kind of based on the robot TCP scaling method vertically reflected |
| CN110370316B (en) * | 2019-06-20 | 2021-12-10 | 重庆大学 | Robot TCP calibration method based on vertical reflection |
| CN112215884A (en) * | 2019-07-09 | 2021-01-12 | 北京地平线机器人技术研发有限公司 | Pose determination method and device for plane marker |
| CN112654886A (en) * | 2020-05-27 | 2021-04-13 | 华为技术有限公司 | External parameter calibration method, device, equipment and storage medium |
| WO2021237520A1 (en) * | 2020-05-27 | 2021-12-02 | 华为技术有限公司 | Method and apparatus for calibrating extrinsics, and device and storage medium |
| CN111652945A (en) * | 2020-06-03 | 2020-09-11 | 北京方程奇迹科技有限公司 | Camera calibration method |
| CN112975959A (en) * | 2021-02-10 | 2021-06-18 | 山东英信计算机技术有限公司 | Machine vision-based radiator assembling and positioning method, system and medium |
| CN113592951A (en) * | 2021-07-14 | 2021-11-02 | 阿波罗智联(北京)科技有限公司 | Method and device for calibrating external parameters of vehicle-road cooperative middle-road side camera and electronic equipment |
| CN116883517A (en) * | 2023-09-07 | 2023-10-13 | 华东交通大学 | Camera parameter calibration method without overlapping view fields based on plane mirror |
| CN116883517B (en) * | 2023-09-07 | 2023-11-10 | 华东交通大学 | Camera parameter calibration method without overlapping view fields based on plane mirror |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109215086A (en) | Camera extrinsic scaling method, equipment and system | |
| WO2021212844A1 (en) | Point cloud stitching method and apparatus, and device and storage device | |
| CN109118545B (en) | Three-dimensional imaging system calibration method and system based on rotating shaft and binocular camera | |
| CN109732228B (en) | Laser galvanometer precision correction method, device and system | |
| CN107492127B (en) | Light field camera parameter calibration method and device, storage medium and computer equipment | |
| JP4914039B2 (en) | Information processing method and apparatus | |
| CN110246185B (en) | Image processing method, device, system, storage medium and calibration system | |
| CN106408556B (en) | A kind of small items measuring system scaling method based on general imaging model | |
| CN104424630A (en) | Three-dimension reconstruction method and device, and mobile terminal | |
| CN106887023A (en) | For scaling board and its scaling method and calibration system that binocular camera is demarcated | |
| CN111220129B (en) | Focusing measurement method with rotating holder and terminal | |
| CN106595528A (en) | Digital speckle-based telecentric microscopic binocular stereoscopic vision measurement method | |
| KR20160116075A (en) | Image processing apparatus having a function for automatically correcting image acquired from the camera and method therefor | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN110490943B (en) | Rapid and accurate calibration method and system of 4D holographic capture system and storage medium | |
| CN102096923A (en) | Fisheye calibration method and device | |
| CN101377405B (en) | Vision measuring method of space round gesture parameter and geometric parameter | |
| WO2018201677A1 (en) | Bundle adjustment-based calibration method and device for telecentric lens-containing three-dimensional imaging system | |
| CN115457147A (en) | Camera calibration method, electronic device and storage medium | |
| CN111699513B (en) | Calibration plate, internal parameter calibration method, machine vision system and storage device | |
| CN113034347B (en) | Oblique photography image processing method, device, processing equipment and storage medium | |
| CN113329179B (en) | Shooting alignment method, device, equipment and storage medium | |
| WO2018233514A1 (en) | Pose measurement method and device, and storage medium | |
| CN109389645B (en) | Camera self-calibration method and system, camera, robot and cloud server | |
| CN115187612A (en) | Plane area measuring method, device and system based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190115 |
|
| RJ01 | Rejection of invention patent application after publication |