CN107067441B - Camera calibration method and device - Google Patents
Camera calibration method and device Download PDFInfo
- Publication number
- CN107067441B CN107067441B CN201710212372.4A CN201710212372A CN107067441B CN 107067441 B CN107067441 B CN 107067441B CN 201710212372 A CN201710212372 A CN 201710212372A CN 107067441 B CN107067441 B CN 107067441B
- Authority
- CN
- China
- Prior art keywords
- point
- neighborhood
- images
- feature
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a camera calibration method and device, and belongs to the technical field of cameras. The method comprises the following steps: for each group of images in the multiple groups of images, acquiring at least eight matching point pairs in one group of images; determining the variance of the gray values of pixel points in the neighborhoods of the two characteristic points included by each matching point pair in the at least eight matching point pairs to obtain at least sixteen variances; screening eight effective point pairs from the at least eight matching point pairs based on the at least sixteen variances; and determining an internal reference matrix of the camera based on the eight effective point pairs corresponding to each group of images in the plurality of groups of images so as to calibrate the camera. The variance of the gray values of the pixel points in the neighborhoods of the two characteristic points can accurately reflect the gray distribution of the pixel points around the two characteristic points, so that the accuracy of the effective point pair determined by the variance of the gray values is improved, and the precision of the calibration result of the camera determined according to the effective point pair is also improved.
Description
Technical Field
The invention relates to the technical field of cameras, in particular to a camera calibration method and device.
Background
In the image measurement process or machine vision application, three-dimensional coordinates of an object in the image in the space can be restored through the image shot by the camera. There is a simple linear relationship between the object in the image and the corresponding object in space, i.e., [ object in image ] ═ K [ object in space ], where K is the geometric model imaged by the camera, i.e., the internal reference matrix used to represent the internal parameters of the camera. In general, the internal reference matrix can be determined by experimental calculation, and the process of determining the internal reference matrix is called calibration of the camera.
In the related art, when the camera calibration is performed, the same object can be photographed from different angles by using the camera, so that a plurality of images can be obtained. And then dividing the plurality of images into a plurality of groups, wherein each group of images comprises two images, and acquiring a plurality of matching point pairs in each group of images by a Harris (Harris) angular point detection method, wherein the matching point pairs are point pairs formed by characteristic points of one point on an object in the space respectively corresponding to the two images in the group of images. After a plurality of matching point pairs are acquired, a base matrix can be determined according to the matching point pairs, wherein the base matrix is used for indicating translation parameters, rotation parameters and internal parameters of the camera in the process of converting from one angle to another angle when the camera shoots the group of images. And determining the intersection point of a connecting line of two points where the optical centers are located and two image planes shot by the camera when the camera shoots two images in the group of images according to the determined basic matrix. For convenience of description, the determined intersection points are referred to as poles, and an internal reference matrix of the camera is determined based on the basis matrix, the poles, and the Kruppa equation.
Since the Harris corner points do not have rotational invariance, if the camera rotates when two images in the group of images are taken, the matching point pairs obtained from the group of images will be inaccurate, and the inaccuracy of the matching point pairs will directly reduce the precision of the camera calibration result.
Disclosure of Invention
In order to solve the problem of reduced precision of a calibration result caused by rotation of a camera in the prior art, the embodiment of the invention provides a camera calibration method and a camera calibration device. The technical scheme is as follows:
in one aspect, a camera calibration method is provided, where the method includes:
for each group of images in a plurality of groups of images, acquiring at least eight matching point pairs in the group of images, wherein the plurality of groups of images are obtained by combining at least three images shot by a camera from different visual angles aiming at the same object in pairs;
determining the variance of gray values of pixel points in neighborhoods of two feature points included by each matching point pair in the at least eight matching point pairs to obtain at least sixteen variances, wherein the size of the neighborhood is a preset size;
screening eight valid point pairs from the at least eight matching point pairs based on the at least sixteen variances;
and determining an internal reference matrix of the camera based on the eight effective point pairs corresponding to each group of images in the plurality of groups of images so as to calibrate the camera.
Optionally, the determining the variance of the gray values of the pixel points in the neighborhood of the two feature points included in each of the at least eight matching point pairs includes:
for each matching point pair, determining the average value of the gray values of the pixel points in the neighborhoods of the two characteristic points included in the matching point pair;
and respectively calculating the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points according to the average value of the gray values of the pixel points in the neighborhoods of the two characteristic points.
Optionally, the determining the variance of the gray values of the pixel points in the neighborhood of the two feature points included in each of the at least eight matching point pairs includes:
for each matching point pair, determining an average value of gray values of pixel points in neighborhoods of a first characteristic point in two characteristic points included in the matching point pair, wherein the first characteristic point is any one of the two characteristic points;
determining the variance of the gray values of the pixel points in the neighborhood of the first feature point based on the gray value of the pixel points in the neighborhood of the first feature point and the average value of the gray values of the pixel points in the neighborhood of the first feature point;
determining the variance of the gray values of the pixel points in the neighborhood of the second feature point based on the gray value of the pixel points in the neighborhood of the second feature point and the average value of the gray values of the pixel points in the neighborhood of the first feature point, wherein the second feature point is the other feature point different from the first feature point in the two feature points.
Optionally, the screening eight valid point pairs from the at least eight matching point pairs based on the at least sixteen variances comprises:
for each matching point pair in the at least eight matching point pairs, calculating the absolute value of the difference between the variances of the gray values of the pixel points in the neighborhoods of the two characteristic points included in the matching point pair;
sequencing the at least eight calculated difference absolute values to obtain a sequencing result;
and based on the sorting result, selecting eight matching point pairs from the at least eight matching point pairs as the eight effective point pairs in the order of the absolute value of the difference from small to large.
In another aspect, a camera calibration apparatus is provided, the apparatus including:
the device comprises an acquisition module, a matching module and a matching module, wherein the acquisition module is used for acquiring at least eight matching point pairs in a group of images for each group of images in the group of images, and the group of images are obtained by pairwise combining at least three images shot by a camera from different visual angles for the same object;
the first determining module is used for determining the variance of the gray values of pixel points in neighborhoods of two feature points included in each matching point pair of the at least eight matching point pairs to obtain at least sixteen variances, and the size of the neighborhood is a preset size;
a screening module for screening eight valid point pairs from the at least eight matching point pairs based on the at least sixteen variances;
and the second determining module is used for determining an internal reference matrix of the camera based on the eight effective point pairs corresponding to each group of images in the multiple groups of images so as to calibrate the camera.
Optionally, the first determining module includes:
the first determining submodule is used for determining the average value of the gray values of pixel points in neighborhoods of two characteristic points included in each matching point pair;
and the first calculation submodule is used for respectively calculating the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points according to the average value of the gray values of the pixel points in the neighborhoods of the two characteristic points.
Optionally, the first determining module includes:
the second determining submodule is used for determining the average value of the gray values of pixel points in the neighborhood of a first characteristic point in two characteristic points included in each matching point pair, wherein the first characteristic point is any one of the two characteristic points;
a third determining submodule, configured to determine a variance of gray values of pixel points in a neighborhood of the first feature point based on a gray value of a pixel point in the neighborhood of the first feature point and an average value of gray values of pixel points in the neighborhood of the first feature point;
a fourth determining submodule, configured to determine a variance of the gray values of the pixel points in the neighborhood of the second feature point based on a gray value of the pixel point in the neighborhood of the second feature point and an average value of the gray values of the pixel points in the neighborhood of the first feature point, where the second feature point is another feature point different from the first feature point in the two feature points.
Optionally, the screening module comprises:
the second calculation submodule is used for calculating the absolute value of the difference between the variances of the gray values of the pixel points in the neighborhoods of the two characteristic points included in the matching point pairs for each matching point pair in the at least eight matching point pairs;
the sorting submodule is used for sorting the at least eight calculated difference absolute values to obtain a sorting result;
and the selecting submodule is used for selecting eight matching point pairs from the at least eight matching point pairs as the eight effective point pairs according to the sequence of the absolute value of the difference from small to large on the basis of the sorting result.
The technical scheme provided by the embodiment of the invention has the following beneficial effects: screening at least eight matching point pairs of a group of images through the variance of gray values of pixel points in neighborhoods of two characteristic points included by each matching point pair in the group of images, thereby obtaining eight effective point pairs of the group of images, and determining a camera calibration result according to the effective point pairs. In the embodiment of the invention, because the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points included in each matching point pair can accurately reflect the gray value distribution of the pixel points around the two characteristic points of the matching point pair, the accuracy of the effective point pair determined by the variance of the gray values is improved, and the precision of the calibration result of the camera determined according to the effective point pair is also improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1A is a flowchart of a camera calibration method according to an embodiment of the present invention;
fig. 1B is a distribution diagram of gray values of pixel points in a neighborhood of feature points included in a matching point pair according to the embodiment of the present invention;
fig. 2A is a schematic structural diagram of a camera calibration device according to an embodiment of the present invention;
fig. 2B is a schematic structural diagram of a first determining module according to an embodiment of the present invention;
fig. 2C is a schematic structural diagram of a first determining module according to an embodiment of the present invention;
fig. 2D is a schematic structural diagram of a screening module according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Before explaining the embodiments of the present invention in detail, an application scenario of the embodiments of the present invention will be described. Camera calibration has received increasing attention as one of the important directions for computer vision technology research. The camera calibration is a process of determining an internal reference matrix K of the camera. The reference matrix K of the camera is as shown in equation (1):
where f is the camera focal length, a is the aspect ratio, s is the tilt factor, (u)
0,v
0) Is the principal point coordinates in the image coordinate system in the image plane. In the ideal case, the principal point sitsThe index may be the center point coordinates of the image plane. However, due to an error in the camera structure, the principal point coordinates may be shifted, that is, in the internal reference matrix, the principal point coordinates are also parameters to be optimally solved. In summary, the reference matrix K includes five unknown parameters.
There are three types of camera calibration methods commonly used at present, which are a traditional calibration method, a calibration method based on active vision, and a self-calibration method. The traditional calibration method usually needs to be performed by means of a specific calibration plate, and although the traditional calibration method is high in calibration result precision, the calibration process is complex and online calibration cannot be performed. The calibration method based on active vision needs to know the motion parameters of the camera to obtain the linear solution of the internal reference matrix of the camera, but the motion of the camera is often uncontrollable, so that the motion parameters of the camera are difficult to determine, and the application range of the calibration method based on active vision is very narrow. In the self-calibration method of the camera, an internal reference matrix of the camera can be determined according to matching point pairs existing in a plurality of images of the same object at different visual angles in space, which are acquired by the camera, without the help of a calibration plate or the need of knowing the motion parameters of the camera.
In the related technology, when matching point pairs are determined, firstly, a plurality of matching point pairs in a group of images are preliminarily determined according to a Harris corner detection method, then, for each matching point pair in the plurality of matching point pairs, neighborhoods of two feature points in the matching point pair can be determined, then, the difference between the gray value of a pixel point in the neighborhood with the preset size of one feature point and the gray value of a corresponding pixel point in the neighborhood with the preset size of the other feature point can be calculated, so that a plurality of gray differences are obtained, and the sum of the gray differences is calculated. And determining the matching point pair corresponding to the smaller sum of the gray differences in the plurality of sums of the gray differences as an effective point pair of the group of images, wherein the effective point pair is an accurate matching point pair corresponding to the group of images.
When the camera calibration is performed by the method, if the camera rotates when two images of the group are shot, the difference between the gray value of the pixel point in the neighborhood of the preset size of one feature point in the matching point pair obtained by calculation and the gray value of the pixel point corresponding to the neighborhood of the preset size of the other feature point cannot reflect whether the gray distribution of the pixel points around the two feature points included in the matching point pair is the same or similar, that is, under the condition that the camera rotates, the accurate effective point pair cannot be determined by the method, so that the precision of the camera calibration result determined according to the effective point pair is reduced.
In order to solve the above problem, an embodiment of the present invention provides a camera calibration method, in which at least eight matching point pairs of a group of images are screened through a variance of gray values of pixel points in neighborhoods of two feature points included in each matching point pair in the group of images, so as to obtain valid point pairs of the group of images, and a camera calibration result is determined according to the valid point pairs. In the embodiment of the invention, because the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points included in each matching point pair can accurately reflect the gray value distribution of the pixel points around the two characteristic points of the matching point pair, the accuracy of the effective point pair determined by the variance of the gray values is improved, and the precision of the calibration result of the camera determined according to the effective point pair is also improved.
Fig. 1A is a flowchart of a camera calibration method according to an embodiment of the present invention, and as shown in fig. 1A, the method includes:
step 101: for each group of images in the multiple groups of images, at least eight matching point pairs in one group of images are obtained, and the multiple groups of images are obtained by pairwise combination of at least three images shot by the camera from different visual angles for the same object.
For an object or a region in space, the camera may capture at least three images from different perspectives, and the camera may send the at least three images to the camera calibration device. When the camera calibration device receives the at least three images, the at least three images can be combined in pairs to obtain a plurality of groups of images. Then, for each group of images in the multiple groups of images, the camera calibration device may preliminarily obtain at least eight matching point pairs in the group of images by using a Harris corner detection method.
For example, assuming that the camera takes three images of the same object from three different perspectives, the camera may send the three images to the camera calibration device. The camera calibration device can combine the three images two by two to obtain a first group of images and a second group of images. Then, the camera calibration apparatus may obtain at least eight first matching point pairs from the first group of images and obtain at least eight second matching point pairs from the second group of images by using a Harris corner detection method.
If the camera calibration device only acquires eight matching point pairs from each group of images, since at least eight matching point pairs are required for calculating the camera internal reference matrix, the camera calibration device does not need to screen the acquired eight matching point pairs, that is, the camera calibration device can directly determine the internal reference matrix of the camera through step 104.
However, in general, there are often more than eight matching point pairs in a group of images preliminarily determined by the Harris corner point detection method, and therefore, the camera calibration apparatus may screen eight effective point pairs from the more than eight matching point pairs through steps 102 and 103, so as to determine the internal reference matrix of the camera according to the screened effective point pairs.
Step 102: determining the variance of the gray values of pixel points in the neighborhoods of the two characteristic points included by each matching point pair in the at least eight matching point pairs to obtain at least sixteen variances, wherein the size of the neighborhoods is a preset size.
After acquiring the at least eight matching point pairs of each group of images in the multiple groups of images, for each matching point pair of the at least eight matching point pairs of each group of images, the camera calibration device may determine the variance of the gray values of the pixel points in the neighborhoods of the two feature points included in each matching point pair in the following two ways.
(1) For each matching point pair, determining the average value of the gray values of the pixel points in the neighborhoods of the two characteristic points included in the matching point pair; and respectively calculating the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points according to the average value of the gray values of the pixel points in the neighborhoods of the two characteristic points.
For each of the two feature points, the camera calibration device may calculate an average value of gray values of pixels in a neighborhood of the feature point according to a formula (2) based on the number of pixels included in the neighborhood of the feature point and a gray value of each pixel.
Wherein, mu
iIs the average value of the gray values of the pixel points in the neighborhood of the feature point, (n)
1,n
2…n
k) The gray value of the pixel points in the feature point is defined, and k is the number of the pixel points included in the neighborhood of the feature point.
After determining the average value of the gray values of the pixel points in the neighborhoods of the two feature points, the camera calibration device may calculate the variance of the gray values of the pixel points in the neighborhood of the feature point through the formula (3) according to the gray value of the pixel point in the neighborhood of each feature point and the average value of the gray values of the pixel points in the neighborhood of the feature point.
Wherein σ
iThe variance of the gray value of the pixel points in the neighborhood of the feature point is obtained.
It should be noted that, when the average value and the variance of the gray values of the pixel points in the neighborhoods of the two feature points are calculated through the above formulas (2) and (3), the pixel points in the neighborhoods of the two feature points may include the two feature points themselves, that is, k may be the number of all the pixel points in the neighborhoods of the feature points including the pixel points themselves. Of course, in the embodiment of the present invention, determining whether the matching point pair is the valid point pair is determined by determining whether the gray distribution around the two feature points of the matching point pair is consistent, so when calculating the variance corresponding to the two feature point pairs, the gray values of the two feature points may not be considered, that is, k may also be the number of all pixel points in the neighborhood of the feature point that do not include the k itself.
For example, as shown in fig. 1B, if the matching point pair is composed of two feature points a and B, and the preset size of the neighborhood of the two feature points of the matching point pair is 5 × 5, then 24 pixel points are included in the neighborhoods of the feature points a and B, respectively. That is, in the formulas (2) and (3), k is 24. In addition, as shown in fig. 1B, the gray value of 24 pixels in the neighborhood of the feature point a is assumed to be (a)
1,a
2…a
24) The gray value of 24 pixels in the neighborhood of the feature point B is (B)
1,b
2…b
24) Then, the camera calibration device can calculate the average value μ of the gray values of 24 pixels in the neighborhood of the feature point a and the feature point B by the formulas (4) and (5)
1And mu
2。
Wherein, mu
1Is the mean value of the gray values, mu, of the pixels in the neighborhood of the feature point A
2Average of gray values of pixels in neighborhood of feature point B, (a)
1,a
2…a
24) Is the gray value of 24 pixel points in the neighborhood of the feature point A, (b)
1,b
2…b
24) The gray values of 24 pixel points in the neighborhood of the feature point B are obtained.
When 24 pixels in the neighborhood of the feature point A and the feature point B are respectively determinedMean value μ of the gray values of the dots
1And mu
2Then, the camera calibration device may determine the variance of the gray values of 24 pixels in the neighborhood of the feature point a and the feature point B by formula (6) and formula (7), respectively.
Wherein σ
1Variance, σ, of gray values of pixels in neighborhood of feature point A
2Is the variance of the gray values of the pixels in the neighborhood of the feature point B.
It should be noted that, in the above example, only the neighborhood with the preset size of 5 × 5 is taken as an example, and no specific limitation on the neighborhood is formed, and the preset size of the neighborhood may also be other values.
When determining the variance of the gray values of the pixel points in the neighborhoods of the two feature points of the matching point pair by the method, the camera calibration device needs to calculate the average value and the variance of the gray values of the pixel points in the neighborhoods of the two feature points respectively, that is, needs to perform two times of average value operation and two times of variance operation. Therefore, in order to reduce the number of operations and enable the two feature points to have correlation in the process of calculating the variance of the gray values corresponding to the two feature points, so as to better reflect the similarity of the two feature points, the embodiment of the present invention may further determine the variance of one feature point by using the average value of the other feature point as a standard, and further obtain the variance of the gray values of the pixel points in the neighborhoods of the two feature points of the matching point pair.
(2) For each matching point pair, determining the average value of the gray values of pixel points in the neighborhoods of the first characteristic point in the two characteristic points included in the matching point pair, wherein the first characteristic point is any one of the two characteristic points; determining the variance of the gray values of the pixel points in the neighborhood of the first feature point based on the gray value of the pixel points in the neighborhood of the first feature point and the average value of the gray values of the pixel points in the neighborhood of the first feature point; and determining the variance of the gray values of the pixel points in the neighborhood of the second characteristic point based on the gray value of the pixel points in the neighborhood of the second characteristic point and the average value of the gray values of the pixel points in the neighborhood of the first characteristic point, wherein the second characteristic point is the other characteristic point which is different from the first characteristic point in the two characteristic points.
Assuming that the matching point pair is a valid point pair, the two feature points included in the matching point pair simultaneously correspond to the same point in space, so that the average of the gray values of the pixel points in the surrounding neighborhood of the same size should be the same regardless of the translation or rotation of the camera. Therefore, the camera calibration device may first calculate the mean value and the variance of the gray values of the pixel points in the neighborhood of one feature point, then use the mean value of the gray values of the pixel points in the neighborhood of the feature point as the mean value of the gray values of the pixel points in the neighborhood of another feature point, and simultaneously calculate the variance of the gray values of the pixel points in the neighborhood of another feature point by combining the gray values of the pixel points in the neighborhood of another feature point. In an ideal situation, if the matching point pair is an effective point pair, then, since the variance of the gray value of the pixel point in the neighborhood of another feature point is calculated by the average value of the feature point, and the gray value of the pixel point in the neighborhood of another feature point should be the same as the gray value of the pixel point in the neighborhood of the feature point, the calculated variance of the gray value of the pixel point in the neighborhood of another feature point will be completely the same as the variance of the gray value of the pixel point in the neighborhood of the feature point. Of course, in practical applications, the variance of two feature points may be different due to the existence of errors. Therefore, the valid point pair can be determined by step 103 according to the magnitude of the difference between the variances of the two feature points.
Still taking the two feature points in the matching point pair shown in fig. 1B as an example, assume that the first feature point is feature point a and the second feature point is feature point B. The camera calibration device can then first map the feature pointsGray value (a) of 24 pixel points in neighborhood of A
1,a
2…a
24) Substituting the gray value of the 24 pixel points in the neighborhood of the characteristic point A into the formula (2) to calculate the average value mu of the gray values of the 24 pixel points
1And according to the gray value (a)
1,a
2…a
24) Average value of μ
1And a formula (3) for calculating the variance sigma of the gray values of 24 pixel points in the neighborhood of the feature point A
1. Then, the camera calibration device can be used for calibrating the camera according to the gray values (B) of 24 pixel points in the neighborhood of the characteristic point B
1,b
2…b
24) And the average value mu of the gray values of 24 pixel points in the neighborhood of the feature point A
1Calculating the variance sigma of the gray values of 24 pixel points in the neighborhood of the feature point B by a formula (3)
2。
After determining the variance of the gray values of the pixel points in the neighborhood of the two feature points included in each matching point pair by the above-mentioned method (1) or (2), since each group of images has at least eight matching point pairs and each matching point pair has two variances, at least sixteen variances can be obtained for each group of images. Then, the camera calibration device may screen eight valid point pairs from the at least eight matching point pairs of each group of images according to the at least sixteen variances. Of course, the camera calibration device may also screen more than eight valid pairs from at least eight matching pairs in each set of images.
Step 103: eight valid point pairs are screened from the at least eight matching point pairs based on the at least sixteen variances.
After obtaining at least sixteen variances, the camera calibration device may calculate a difference absolute value between the variances of the gray values of the pixel points in the neighborhoods of the two feature points included in each first matching point pair of the at least eight matching point pairs; sequencing the at least eight calculated difference absolute values to obtain a sequencing result; based on the sorting result, from among the at least eight pairs of matching points, eight pairs of matching points are selected as eight valid pairs in the order of the absolute value of the difference from small to large.
Based on the description in step 102, when two feature points in the matching point pair correspond to the same point in the space, ideally, the gray level distributions of the pixel points in the neighborhoods of the two feature points should be the same, that is, the variances of the gray levels of the pixel points in the neighborhoods of the two feature points should be completely equal. Considering the existence of errors in practical application, the variance of the gray values of the pixel points in the neighborhoods of the two feature points may have a certain difference, but the difference is extremely small. On the contrary, if the two feature points in the matching point pair correspond to different points in the space, that is, the matching point pair is not a valid point pair, the gray level distribution of the pixel points in the neighborhoods of the two feature points inevitably has a great difference, that is, the variance of the gray level values of the pixel points in the neighborhoods of the two feature points has a great difference. Therefore, the camera calibration device can screen at least eight matching point pairs through the difference absolute value of the variance of the gray values of the pixel points in the neighborhoods of the two feature points in the matching point pairs, so as to determine the effective point pairs.
For a plurality of groups of images acquired by the camera, at least sixteen variances correspond to at least eight matching point pairs of each group of images. For each matching point pair of the set of images, an absolute value of a difference between two variances for each matching point pair may be calculated. Then, the camera calibration device may sort the at least eight calculated absolute difference values from small to large to obtain a sorting result. Since the smaller the absolute value of the difference is, the higher the matching degree of the two feature points in the matching point pair is, that is, the higher the accuracy of the matching point pair being an effective point pair is, the camera calibration device may determine the matching point pair corresponding to the absolute value of the difference arranged in the first eight in the sorting result as eight effective point pairs. Of course, if the number of valid point pairs to be obtained is greater than eight, for example, ten valid point pairs need to be obtained, the matching point pair corresponding to the absolute value of the difference value of the top ten in the sorting result is determined as the valid point pair.
Step 104: and determining an internal reference matrix of the camera based on the eight effective point pairs so as to calibrate the camera.
For each of the plurality of sets of images acquired by the camera, eight valid point pairs corresponding to the set of images can be determined in step 103. After the camera obtains eight effective point pairs corresponding to each group of images, a base matrix corresponding to the group of images can be obtained by calculation through a formula (8) according to coordinates of two feature points included in each effective point pair in an image coordinate system of an image plane corresponding to each effective point pair.
(m')
TFm＝0 (8)
Wherein m 'and m are homogeneous coordinates of two characteristic points included in the effective point pair, (m')
TFor the transpose of m', F is the base matrix containing 7 independent variables.
According to at least eight matching point pairs of each group of images, a basic matrix F can be obtained through calculation, and when the camera acquires multiple groups of images, a plurality of basic matrices F can be obtained through corresponding calculation. After determining the plurality of basis matrices F, a plurality of poles e can be calculated by equation (9).
Fe＝0 (9)
After the pole e and the basis matrix F are determined, ω can be calculated according to the Kruppa equation of equation (10).
Wherein ω is KK
TK is the internal reference matrix of the camera, F
TIs a transposed matrix of the base matrix F, [ e ]]
×The anti-symmetric matrix of the poles e,
is [ e ]]
×The transposed matrix of (2).
It should be noted that ω is a 3 × 3 matrix containing five unknown parameters, and the internal parameters of the camera are not constant because of the influence of temperature variation in the environment and other factors on the internal parameters of the camera, in this case, two Kruppa equations can be obtained by one set of images, and two independent equations containing the unknown parameters of ω can be obtained according to each Kruppa equation, so that when determining the five unknown parameters of ω,at least three Kruppa equations are needed, namely, the camera at least needs to shoot from different visual angles to obtain three images, and at least two groups of images are obtained by combining the three images in pairs. Then, ω is determined according to the at least three Kruppa equations. After ω is determined, by the square root method (Cholesky), according to ω KK
TThe internal reference matrix K of the camera can be determined.
In the embodiment of the invention, the camera calibration device can screen at least eight matching point pairs of a group of images through the variance of gray values of pixel points in neighborhoods of two feature points included in each matching point pair in the group of images, so as to obtain effective point pairs of the group of images, and determine the camera calibration result according to the effective point pairs. In the embodiment of the invention, because the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points included in each matching point pair can accurately reflect the gray value distribution of the pixel points around the two characteristic points of the matching point pair, the accuracy of the effective point pair determined by the variance of the gray values is improved, and the precision of the calibration result of the camera determined according to the effective point pair is also improved.
Fig. 2A is a schematic structural diagram of a camera calibration apparatus 200 according to an embodiment of the present invention, and as shown in fig. 2A, the apparatus 200 includes:
an obtaining module 201, configured to obtain, for each group of images in a plurality of groups of images, at least eight matching point pairs in the group of images, where the plurality of groups of images are obtained by combining, in pairs, at least three images captured by a camera from different viewing angles for a same object;
a first determining module 202, configured to determine a variance of gray values of pixel points in neighborhoods of two feature points included in each of the at least eight matching point pairs, to obtain at least sixteen variances, where a size of a neighborhood is a preset size;
a screening module 203, configured to screen eight valid point pairs from the at least eight matching point pairs based on the at least sixteen variances;
the second determining module 204 is configured to determine an internal reference matrix of the camera based on the eight valid point pairs corresponding to each group of images in the multiple groups of images, so as to calibrate the camera.
Optionally, referring to fig. 2B, the first determining module 202 includes:
the first determining submodule 2021 is configured to determine, for each matching point pair, an average value of gray values of pixel points in neighborhoods of two feature points included in the matching point pair;
the first calculating submodule 2022 is configured to calculate variances of the gray values of the pixel points in the neighborhoods of the two feature points respectively according to the average values of the gray values of the pixel points in the neighborhoods of the two feature points.
Optionally, referring to fig. 2C, the first determining module 202 includes:
the second determining submodule 2023 is configured to determine, for each matching point pair, an average value of gray values of pixel points in a neighborhood of a first feature point of the two feature points included in the matching point pair, where the first feature point is any one of the two feature points;
the third determining submodule 2024 is configured to determine a variance of the gray values of the pixel points in the neighborhood of the first feature point based on the gray value of the pixel point in the neighborhood of the first feature point and an average value of the gray values of the pixel points in the neighborhood of the first feature point;
the fourth determining submodule 2025 is configured to determine a variance of the gray values of the pixel points in the neighborhood of the second feature point based on the gray value of the pixel point in the neighborhood of the second feature point and the average value of the gray values of the pixel points in the neighborhood of the first feature point, where the second feature point is another feature point different from the first feature point in the two feature points.
Optionally, referring to fig. 2D, the screening module 203 comprises:
the second calculating submodule 2031 is configured to calculate an absolute value of a difference between variances of gray values of pixel points in neighborhoods of two feature points included in each of the at least eight matching point pairs;
the sorting submodule 2032 is configured to sort the at least eight calculated absolute difference values to obtain a sorting result;
the selecting sub-module 2033 is configured to select eight matching point pairs as eight valid point pairs from at least eight matching point pairs in order of smaller absolute value of the difference to larger absolute value of the difference based on the sorting result.
In summary, in the embodiment of the present invention, at least eight matching point pairs of a group of images are screened according to the variance of the gray values of the pixel points in the neighborhoods of the two feature points included in each matching point pair in the group of images, so as to obtain eight effective point pairs of the group of images, and a camera calibration result is determined according to the eight effective point pairs. In the embodiment of the invention, because the variance of the gray values of the pixel points in the neighborhoods of the two characteristic points included in each matching point pair can accurately reflect the gray value distribution of the pixel points around the two characteristic points of the matching point pair, the accuracy of the effective point pair determined by the variance of the gray values is improved, and the precision of the calibration result of the camera determined according to the effective point pair is also improved.
It should be noted that: in the camera calibration device provided in the above embodiment, only the division of the above functional modules is used for illustration when performing camera calibration, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above described functions. In addition, the camera calibration device and the camera calibration method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (4)
1. A camera calibration method, the method comprising:
for each group of images in a plurality of groups of images, acquiring at least eight matching point pairs in one group of images, wherein the plurality of groups of images are obtained by pairwise combination of at least three images shot by a camera from different visual angles for the same object;
determining the variance of gray values of pixel points in neighborhoods of two feature points included by each matching point pair in the at least eight matching point pairs to obtain at least sixteen variances, wherein the size of the neighborhood is a preset size;
screening eight valid point pairs from the at least eight matching point pairs based on the at least sixteen variances;
determining an internal reference matrix of the camera based on eight effective point pairs corresponding to each group of images in the multiple groups of images so as to calibrate the camera;
wherein, the determining the variance of the gray values of the pixel points in the neighborhoods of the two feature points included in each matching point pair of the at least eight matching point pairs comprises:
for each matching point pair, determining an average value of gray values of pixel points in neighborhoods of a first characteristic point in two characteristic points included in the matching point pair, wherein the first characteristic point is any one of the two characteristic points;
determining the variance of the gray values of the pixels in the neighborhood of the first feature point based on the gray value of the pixel in the neighborhood of the first feature point and the average value of the gray values of the pixels in the neighborhood of the first feature point, wherein the pixels in the neighborhood of the first feature point do not include the first feature point;
determining the variance of the gray values of the pixels in the neighborhood of the second feature point based on the gray value of the pixels in the neighborhood of the second feature point and the average value of the gray values of the pixels in the neighborhood of the first feature point, wherein the second feature point is another feature point different from the first feature point in the two feature points, and the pixels in the neighborhood of the second feature point do not include the second feature point.
2. The method of claim 1, wherein said screening eight valid point pairs from said at least eight matching point pairs based on said at least sixteen variances comprises:
for each matching point pair in the at least eight matching point pairs, calculating the absolute value of the difference between the variances of the gray values of the pixel points in the neighborhoods of the two characteristic points included in the matching point pair;
sequencing the at least eight calculated difference absolute values to obtain a sequencing result;
and based on the sorting result, selecting eight matching point pairs from the at least eight matching point pairs as the eight effective point pairs in the order of the absolute value of the difference from small to large.
3. A camera calibration apparatus, the apparatus comprising:
the device comprises an acquisition module, a matching module and a matching module, wherein the acquisition module is used for acquiring at least eight matching point pairs in a group of images for each group of images in a plurality of groups of images, and the plurality of groups of images are obtained by combining at least three images shot by a camera from different visual angles aiming at the same object in pairs;
the first determining module is used for determining the variance of the gray values of pixel points in neighborhoods of two feature points included in each matching point pair of the at least eight matching point pairs to obtain at least sixteen variances, and the size of the neighborhood is a preset size;
a screening module for screening eight valid point pairs from the at least eight matching point pairs based on the at least sixteen variances;
the second determining module is used for determining an internal reference matrix of the camera based on the eight effective point pairs corresponding to each group of images in the multiple groups of images so as to calibrate the camera;
the first determining module includes:
the second determining submodule is used for determining the average value of the gray values of pixel points in the neighborhood of a first characteristic point in two characteristic points included in each matching point pair, wherein the first characteristic point is any one of the two characteristic points;
a third determining submodule, configured to determine a variance of gray values of pixel points in a neighborhood of the first feature point based on a gray value of a pixel point in the neighborhood of the first feature point and an average value of gray values of pixel points in the neighborhood of the first feature point, where the pixel points in the neighborhood of the first feature point do not include the first feature point;
a fourth determining submodule, configured to determine a variance of the gray values of the pixels in the neighborhood of the second feature point based on a gray value of a pixel in the neighborhood of the second feature point and an average value of the gray values of the pixels in the neighborhood of the first feature point, where the second feature point is another feature point different from the first feature point in the two feature points, and the pixel in the neighborhood of the second feature point does not include the second feature point.
4. The apparatus of claim 3, wherein the screening module comprises:
the second calculation submodule is used for calculating the absolute value of the difference between the variances of the gray values of the pixel points in the neighborhoods of the two characteristic points included in the matching point pairs for each matching point pair in the at least eight matching point pairs;
the sorting submodule is used for sorting the at least eight calculated difference absolute values to obtain a sorting result;
and the selecting submodule is used for selecting eight matching point pairs from the at least eight matching point pairs as the eight effective point pairs according to the sequence of the absolute value of the difference from small to large on the basis of the sorting result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710212372.4A CN107067441B (en) | 2017-04-01 | 2017-04-01 | Camera calibration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710212372.4A CN107067441B (en) | 2017-04-01 | 2017-04-01 | Camera calibration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107067441A CN107067441A (en) | 2017-08-18 |
| CN107067441B true CN107067441B (en) | 2020-02-11 |
Family
ID=59602773
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710212372.4A Active CN107067441B (en) | 2017-04-01 | 2017-04-01 | Camera calibration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107067441B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275765B (en) * | 2018-12-05 | 2023-09-05 | 杭州海康威视数字技术股份有限公司 | Method and device for determining target GPS and camera |
| CN111243035B (en) * | 2020-04-29 | 2020-08-14 | 成都纵横自动化技术股份有限公司 | Camera calibration method and device, electronic equipment and computer-readable storage medium |
| CN112581546A (en) * | 2020-12-30 | 2021-03-30 | 深圳市杉川机器人有限公司 | Camera calibration method and device, computer equipment and storage medium |
| CN113177989A (en) * | 2021-05-07 | 2021-07-27 | 深圳云甲科技有限公司 | Intraoral scanner calibration method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750704A (en) * | 2012-06-29 | 2012-10-24 | 吉林大学 | Step-by-step video camera self-calibration method |
| CN103176607A (en) * | 2013-04-16 | 2013-06-26 | 重庆市科学技术研究院 | Eye-controlled mouse realization method and system |
| CN104268880A (en) * | 2014-09-29 | 2015-01-07 | 沈阳工业大学 | Depth information obtaining method based on combination of features and region matching |
-
2017
- 2017-04-01 CN CN201710212372.4A patent/CN107067441B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750704A (en) * | 2012-06-29 | 2012-10-24 | 吉林大学 | Step-by-step video camera self-calibration method |
| CN103176607A (en) * | 2013-04-16 | 2013-06-26 | 重庆市科学技术研究院 | Eye-controlled mouse realization method and system |
| CN104268880A (en) * | 2014-09-29 | 2015-01-07 | 沈阳工业大学 | Depth information obtaining method based on combination of features and region matching |
Non-Patent Citations (2)
| Title |
|---|
| 双目视觉技术在结构件尺寸测量中的应用;侯跃谦 等;《计算机应用与软件》;20130930;137-140页 * |
| 基于主动视觉的摄像机自标定方法;于洪川 等;《机器人》;19991231;1-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107067441A (en) | 2017-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10924729B2 (en) | Method and device for calibration | |
| CN111179358B (en) | Calibration method, device, equipment and storage medium | |
| CN106651942B (en) | Three-dimensional rotating detection and rotary shaft localization method based on characteristic point | |
| CN107194972B (en) | Camera calibration method and system | |
| CN111210468B (en) | Image depth information acquisition method and device | |
| CN107067441B (en) | Camera calibration method and device | |
| US10726580B2 (en) | Method and device for calibration | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN108629810B (en) | Calibration method and device of binocular camera and terminal | |
| KR101759798B1 (en) | Method, device and system for generating an indoor two dimensional plan view image | |
| CN112802124A (en) | Calibration method and device for multiple stereo cameras, electronic equipment and storage medium | |
| CN112132908A (en) | Camera external parameter calibration method and device based on intelligent detection technology | |
| CN110738703A (en) | Positioning method and device, terminal and storage medium | |
| CN112489137A (en) | RGBD camera calibration method and system | |
| CN114299156A (en) | Method for calibrating and unifying coordinates of multiple cameras in non-overlapping area | |
| CN112229323A (en) | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method | |
| CN115830135A (en) | Image processing method and device and electronic equipment | |
| CN114998773A (en) | Characteristic mismatching elimination method and system suitable for aerial image of unmanned aerial vehicle system | |
| CN110838146A (en) | Homonymy point matching method, system, device and medium for coplanar cross-ratio constraint | |
| CN116597246A (en) | Model training method, target detection method, electronic device and storage medium | |
| CN112489141B (en) | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera | |
| Tagoe et al. | Determination of the Interior Orientation Parameters of a Non-metric Digital Camera for Terrestrial Photogrammetric Applications | |
| CN115018922A (en) | Distortion parameter calibration method, electronic device and computer readable storage medium | |
| CN114897990A (en) | Camera distortion calibration method and system based on neural network and storage medium | |
| CN110930344B (en) | Target quality determination method, device and system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |