CN102750698B - Texture camera calibration device, texture camera calibration method and geometry correction method of texture image of texture camera - Google Patents

Abstract

The invention relates to a texture camera calibration device and method applied to optical three-dimensional measurement and a texture image geometric correction method thereof. The texture camera calibration setup consists of a reference plate with a dot marker array and a moving stage. The reference plate is fixed on the mobile workbench, perpendicular to its moving direction. The texture camera calibration method is as follows: the mobile worktable drives the reference plate to move in parallel to at least 3 positions, and reads the moving distance as the Z coordinate of the circle center of the dot array on it; at each position of the reference plate, the texture camera captures the dot mark array image, Extract the pixel coordinates of the center of the dot; solve 14 undetermined parameters. The texture image geometric correction method is to generate a blank measurement camera perspective texture image; read the three-dimensional coordinates of the measured object surface points corresponding to each pixel of the blank image, and solve its corresponding pixel coordinates in the color texture image captured by the texture camera, And fill the blank image pixels of the measurement camera perspective with its color.

Description

Texture camera calibration device and method and texture image geometric correction method

technical field

The invention relates to a camera calibration device and method and its image geometry correction method, in particular to a texture camera calibration device and method in optical three-dimensional measurement and its texture image geometry correction method.

Background technique

Optical three-dimensional measurement technology refers to the technology of measuring the surface shape of an object by means of optical methods and obtaining three-dimensional point cloud data of the object surface. With its non-contact advantages, this technology has very broad application prospects in industrial inspection, biology and medicine, protection of human cultural heritage, and computer vision. Active optical three-dimensional measurement technology uses active lighting sources to project structured light coding patterns to objects, and uses cameras to capture deformed patterns from another angle, from which the three-dimensional point cloud data of the object surface is calculated. Passive optical three-dimensional measurement technology uses environmental lighting, uses two or more cameras to shoot objects from multiple angles, and uses the principle of stereo vision to reconstruct three-dimensional point cloud data of the object surface. The 3D model of the object can be reconstructed by using the measured 3D point cloud data. In the measurement system, in order to avoid the gray scale distortion caused by color correction or color balance operation to reduce the measurement accuracy, a black and white camera is generally used as the measurement camera, and the reconstructed 3D point cloud data has a direct correspondence with the measurement camera pixels.

In order to enhance the visual realism when displaying the 3D measurement results, the photo of the measured 3D object can be mapped to the surface of the reconstructed 3D model through texture mapping. In optical 3D measurement, there are three main ways to obtain texture photos. One is to directly take a photo of the texture of the object with a measuring camera. In this method, point cloud data and texture data are obtained by the same camera, which can be accurately registered, and there is no position error when mapping. However, since the measurement camera is generally a black and white camera, the color information is lost. If a color camera is used for measurement (S Zhang and S-T Yau, "Simultaneous three-dimensional geometry and color texture acquisition using a single-chip color camera," Opt. Eng. 47(12), 123604, 2008), the Bayer color filter And the camera color balance calculation will cause grayscale distortion, thereby reducing the measurement accuracy. Second, the light beam reflected by the object surface is divided into two paths by using a beam splitting device. In one of them, texture photos were taken with a color camera (S. Zhang and P. S. Huang, “High-resolution, real-time threedimensional shape measurement,” Opt. Eng. 45, 123601, 2006). This method not only needs to increase the spectroscopic device, but also needs to accurately correspond the pixels of the texture camera to the pixels of the measurement camera, and the adjustment is difficult. Third, the color texture camera is installed at any position next to the measurement camera. This method is simple and easy, but due to the difference in the perspective of the texture camera and the measurement camera. Texture photos taken directly cannot be directly used for texture mapping. It is necessary to transform the texture photo to the viewing angle of the measurement camera through image registration or geometric correction to achieve texture mapping.

The above third method is simple and practical, but it is necessary to establish the mapping relationship between the texture camera perspective image and the measurement camera perspective image through calibration, and then use the mapping relationship to transform the color texture image captured by the texture camera to the measurement camera The angle of view. Since there is a direct correspondence between the measured camera pixels and the reconstructed point cloud, the transformed image can be used for texture mapping. But general camera calibration methods (such as Z. Zhang, "Flexibel camera calibration by viewing a plane from unknown orientation," International conference on computer vision (ICCV'99), Corfu, Greece. 666-673,1999) are suitable for obtaining a The internal and external parameters of the camera cannot be directly used to determine the mapping relationship between the image pixels of the above two perspectives.

Contents of the invention

The object of the present invention is to provide a texture camera calibration device and method for optical three-dimensional measurement and a geometric correction method for texture images. Through the above device and method, the pixel mapping relationship between the texture camera perspective image and the measurement camera perspective image in the optical three-dimensional measurement system can be determined. Through the mapping relationship, the color texture image captured by the texture camera can be converted to the measurement camera perspective, thereby realizing the texture mapping of the 3D model and enhancing the visual reality when displaying the 3D measurement results.

To achieve the above object, the present invention adopts following scheme:

A texture camera calibration device in optical three-dimensional measurement, comprising a reference plate with a two-dimensional dot mark array on the surface and a mobile workbench. The dot mark array on the reference plate is distributed along two vertical directions of X and Y. The center coordinates ( X _k , Y _k ) of each dot are known precisely, where k ( k =1,2,…, K ) is the serial number of the dot. In order to improve the contrast of the image, the dot mark and the background color are black and white respectively. The reference plate is fixed on the mobile worktable, and the plane of the reference plate is perpendicular to the moving direction of the mobile worktable, so the mobile worktable can drive the reference plate to move along the Z-axis direction, and the moving distance can be accurately read.

A texture camera calibration method in optical three-dimensional measurement, the calibration process includes the following steps:

S1: Put the above-mentioned calibration device in the measurement system, and the dot mark on the reference plate faces the direction of the camera, which can be photographed by two cameras at the same time. The initial position of the reference plate is recorded as Z ₀ =0 as the reference position for the measured depth map. The measurement camera and the texture camera simultaneously capture images of the array of marked dots. Move the table to drive the reference plate to N different positions ( N ≥ 2), and record its position Z _n ( n =1, 2, ..., N ). Through this process, the center of the marked circle of the reference plate constitutes a three-dimensional space lattice, and the space coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) of each point are known. At each position reached by the movement of the reference plate, an image of the array of marked dots is taken with a texture camera.

S2: Process the image obtained by the texture camera in step S1, and extract the centroid of each dot as the pixel coordinates ( u _{k, n} , v _{k, n} ) of the dot center.

S3: Substitute the pixel coordinates ( u _{k, n} , v _{k, n} ) of the center of each dot in the texture camera image and the corresponding three-dimensional space point coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) into the formula :

Form a system of equations containing 2 K ( N+1 ) equations. Using the method of least squares, 14 unknown and undetermined parameters q _m (m=1,2,...,14) can be solved from this equation system. These parameters determine the mapping relationship between the texture camera perspective image and the measurement camera perspective image pixels in the optical three-dimensional measurement system.

A geometric correction method for texture images in optical three-dimensional measurement. In actual measurement, the measurement system can measure the three-dimensional point cloud data of the object surface, where the three-dimensional coordinates of the object surface point corresponding to the measurement camera pixel ( s , t ) are [ X ( s , t ), Y ( s , t ), Z ( s , t )]. At the same time, the color texture image of the object surface can be captured by the texture camera. The red, green and blue primary color components of the ( u , v ) pixel on the texture image are [ RT ₍ u , v ), G _T ( u , v ), BT ₍ u , v )]. Using the calibration results of the aforementioned calibration method, the color image can be geometrically corrected, and it can be transformed from the texture camera perspective image to the measurement camera perspective. The steps are as follows:

S4: Generate a blank color image [ RM ₍ s , t ), G _M ( s , t ), B _M ( s , t )] that measures the camera’s viewing angle, and its components represent the pixels in the image ( s , t ) of the red, green and blue primary color components.

S5: For each pixel coordinate ( s , t ) of the above blank image, use the formula

Calculate its corresponding pixel coordinates ( u , v ) in the texture camera image.

S6: Read each color component of the pixel ( u , v ) in the texture camera image. Fill the blank color image ( s , t ) pixels measuring the camera viewing angle with these color components, i.e.

Perform the same operation on all pixels ( s , t ) to obtain a color texture image measuring the camera's viewing angle, and realize the geometric correction of the texture image.

Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

1. The present invention can ensure that the process of point cloud measurement and texture acquisition in optical three-dimensional measurement is carried out independently, avoiding mutual interference. That is, black and white cameras are used for measurement to ensure measurement accuracy, and color cameras are used to record texture images to preserve color information.

2. The present invention allows the texture camera to be installed at any position next to the measurement camera without adding a spectroscopic device, which is easy to implement.

3. The selection of the parameters of the texture camera can be different from that of the measurement camera. For example, a measurement camera with high quantization resolution and high linearity can be used for measurement to reduce the impact of quantization errors and nonlinear responses on measurement accuracy; at the same time, a general quantization resolution (8-bit) and a color camera with GAMMA correction can be selected Shoot color texture images to ensure the visual effect of texture images. The image sizes of the two can also be different.

4. The device of the present invention is simple, the method is convenient to operate, and there is no need to obtain the internal and external parameters of the texture camera. The calibration device can be used for calibration of the measurement camera and the measurement system at the same time. Texture camera calibration and measurement system calibration can be performed at the same time, which is beneficial to reduce calibration time.

Description of drawings

Fig. 1 is a structural schematic diagram of the "texture camera calibration device in optical three-dimensional measurement" of the present invention;

Fig. 2 is a schematic diagram of the "texture camera calibration method in optical three-dimensional measurement" of the present invention.

Detailed ways

Preferred embodiments of the present invention are described in detail as follows in conjunction with the accompanying drawings:

Embodiment 1: Referring to Fig. 1, the texture camera calibration device used in optical three-dimensional measurement is composed of a reference plate (1) and a mobile worktable (2); the surface of the reference plate (1) has a two-dimensional dot mark array , the dot marker array is distributed along the two vertical directions of X and Y ; the center coordinates ( X _k , Y _k ) of each dot on the reference plate (1) are known precisely, where k ( k =1,2,…, K ) It is the serial number of the dot; the dot mark and the background color on the reference plate (1) are black and white respectively, so as to improve the contrast of its image and facilitate the segmentation of the dot mark; the reference plate (1) is fixed on the mobile workbench (2 ), and make the plane of the reference plate (1) perpendicular to the moving direction of the mobile worktable (2); the mobile workbench (2) can drive the reference plate (1) to move along the Z-axis direction, and the moving distance can be read accurately; the calibration device During calibration, it is placed in the measurement space of the measurement system composed of the structured light source (3) and the black-and-white measurement camera (5), and is located in the field of view of the black-and-white measurement camera (5) and the color texture camera.

Embodiment 2: Referring to FIG. 2 , the texture camera calibration method applied in optical three-dimensional measurement is calibrated using the device of Embodiment 1. The calibration principle and steps are introduced as follows.

Point the dot mark on the reference plate towards the direction of the camera, which can be captured by two cameras at the same time.

The reference plate can be used for the calibration of the measurement system and the calibration of the texture camera at the same time. The measurement camera in the measurement system can capture the reference plate circle point mark array to establish the mapping relationship between the pixel coordinates of the measurement camera and the three-dimensional coordinates of the surface points of the measured object, that is, to realize the lateral ( X and Y direction) calibration of the measurement system . At the same time, through the translation of the reference plate, the structured light source is used to project the structured light pattern to the reference plate, and the measurement camera obtains the deformed structured light pattern when the reference plate is located at different depth positions. By analyzing these patterns, the mapping relationship between the deformation of the structured light pattern and the depth of the surface point of the measured object can be established, that is, the longitudinal ( Z direction) calibration of the measurement system can be realized. The present invention is mainly aimed at the calibration problem of the texture camera. Since the calibration of the measurement system and the calibration of the texture camera can be performed simultaneously, the measurement system and the texture shooting system have the same spatial coordinate system.

The initial position of the reference plate is recorded as Z ₀ =0 as the reference position for the measured depth map. The measurement camera and the texture camera simultaneously capture images of the array of marked dots. Move the table to drive the reference plate to N different positions ( N ≥ 2), and record its position Z _n ( n =1, 2, ..., N ). Through this process, the center of the marked circle of the reference plate constitutes a three-dimensional space lattice, and the space coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) of each point are known. At each position reached by the movement of the reference plate, an image of the array of marked dots is taken with a texture camera.

Process the marked origin image obtained by the texture camera. First, the noise of the marked origin image can be removed by smoothing and filtering, and then the image is segmented for each dot. Since the reference plate is a plane shape, the illumination on it is relatively uniform, and the dot mark and the background color are black and white, and the contrast is relatively high. Image segmentation can be realized by simply setting a global threshold. The image is binarized by segmentation, marking the origin as white for 1 and the background as black and white for 0. Then find the centroid of each dot as the pixel coordinates ( u _{k, n} , v _{k, n} ) of the dot center:

Among them, the subscript k ( k =1,2,…, K ) indicates the serial number of the marked dot on the reference plate, n ( n =0, 1, 2, …, N ) indicates the serial number of the reference plate position; D _k,n is Referring to the connected area of the k -th marked dot at the n -th position of the plate, ( i , j ) is the pixel coordinate of the binary image, and the denominator of the above formula is actually the area of the area D _k,n .

As mentioned above, the mobile workbench drives the reference plate to move, and the center of the marked circle on the reference plate forms a three-dimensional space lattice, and the space coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) of each point are Know. Among them ( X _{k, n} , Y _{k, n} ) are the coordinates of the center of the dot mark, which have been determined when the reference plate is made. The coordinates Z _{k, n} are determined by the moving position of the reference plate and are read while the reference plate is moving.

Substitute the pixel coordinates ( u _{k, n} , v _{k, n} ) of the center of each dot in the texture camera image and their corresponding three-dimensional space point coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) into the formula:

Form a system of equations containing 2 K ( N+1 ) equations. The above formula is obtained through rigorous geometric analysis of the measurement system, which can accurately describe the mapping relationship between the coordinates of a point in the measurement space and the camera pixel coordinates. Note that the above equations are nonlinear and need to be solved by an iterative method, which is difficult, and can be transformed into the following linear equations

where T denotes matrix transpose. Using the Housholder transformation, the least squares solution of this linear equation system can be solved, and 14 unknown parameters q _m (m=1,2,...,14) can be obtained. These parameters determine the mapping relationship between the texture camera perspective image in the optical three-dimensional measurement system and the three-dimensional coordinates of the object surface points in the measurement space. Since there is a definite direct correspondence between the object surface point cloud and the measurement camera pixels, the parameters obtained from the above calibration can be used to determine the mapping relationship between the texture camera perspective image and the measurement camera perspective image pixels.

Embodiment 3: The geometric correction method applied to the texture image in optical three-dimensional measurement uses the calibration result of Embodiment 2 to perform geometric correction on the color texture image captured by the texture camera, and convert it into the measurement camera perspective image, so as to realize the measurement The texture map for the resulting 3D model.

In actual measurement, the measurement system can measure the three-dimensional point cloud data of the object surface, and there is a direct correspondence between the point cloud data and the measurement camera pixels, and the three-dimensional coordinates of the object surface point corresponding to the pixel ( s , t ) can be expressed as [ X ( s , t ), Y ( s , t ), Z ( s , t )]. While measuring, the texture camera can capture the texture image of the object surface. The texture image is a color image, which can be represented by models such as RGB model or HIS. Using the RGB model, the red, green and blue primary color components of ( u , v ) pixels on the texture image can _be expressed as [ RT ( u , v ), G _T ( u , v ), B _T ( u , v )] .

In order to obtain the texture image of the measurement camera angle of view, a blank color image of the angle of view can be generated first, and the RGB model is also used. The red, green, and blue primary color components of the image pixel ( s , t ) are [ R _M ( s , t ), G _M ( s , t ), B _M ( s , t )].

For each pixel coordinate ( s , t ) of the blank image above, using the calibration result of Embodiment 2, that is, the parameter q _m (m=1,2,...,14), through the formula

Its corresponding pixel coordinates ( u , v ) in the texture camera image can be calculated.

Read the individual color components of a pixel ( u , v ) in the texture camera image. Fill the blank color image ( s , t ) pixels measuring the camera viewing angle with these color components, i.e.

In this way, the color texture image for measuring the viewing angle of the camera can be obtained, and the geometric correction of the texture image is realized.

This method does not require the texture camera to have the same pixel size and number of pixels as the measurement camera. In the above process, the pixel coordinates ( u , v ) of the texture image calculated by ( s , t ) are generally non-integer values, which can be rounded to integer values by rounding, or by using the bilinear difference method ( u , v ) Calculate the color gray value of the nearest 4 pixels around; In addition, it is also possible that the texture image pixel coordinates ( u , v ) calculated by ( s , t ) exceed the size range of the texture image. At this time, the color texture of the pixel cannot be obtained information.

Claims (3)

1. a texture camera calibration method is characterized in that the concrete steps of the method are: The first step is to direct the dot mark of the reference plate (1) towards the direction of the measurement camera (4) and the texture camera (5), which can be photographed by the two cameras (4, 5) simultaneously; the initial position of the reference plate (1) is recorded as Z ₀ =0; use the texture camera (5) to capture the image of the marked dot array; move the worktable (2) to drive the reference plate (1) to N different positions, N ≥ 2, and record its position Z _n , n = 1, 2, ..., N ; Through this process, the center of the marked circle of the reference plate (1) constitutes a three-dimensional space lattice, and the space coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) of each point are known, where k is the circle Point marking serial number; At each moving arrival position of the reference plate (1), the image of the marking circle point array is photographed with a texture camera (5); The second step is to process the image obtained by the texture camera (5) in the first step, and extract the centroid of each dot as the pixel coordinates ( u _{k, n} , v _{k, n} ) of the dot center; The third step, the pixel coordinates ( u _{k, n} , v _{k, n} ) of the center of each dot in the texture camera image and the corresponding three-dimensional space point coordinates ( X _{k, n} , Y _{k, n} , Z _{k, n} ) Into the formula: A system of equations containing 2 K ( N+1 ) equations is formed; 14 unknown and undetermined parameters q _m can be obtained from this system of equations by using the least square method, m=1,2,…,14; these parameters determine The mapping relationship between texture camera perspective image and measurement camera perspective image pixel in optical three-dimensional measurement system. 2. A texture camera calibration device, for realizing the texture camera calibration method according to claim 1, characterized in that the device is made up of a reference flat panel (1) and a mobile workbench (2); on the reference flat panel (1) There is a two-dimensional array of dot marks distributed along the two vertical directions of X and Y ; the coordinates ( X _k , Y _k ) of the center of the dot mark on the reference plate (1) are precisely known, where k is the serial number of the dot mark; The dot mark and the background color on the reference plate (1) are black and white respectively, to improve the contrast of its image; the reference plate (1) is fixed on the mobile workbench (2), and the reference plate (1) plane and The moving direction of the mobile worktable (2) is vertical; the mobile worktable (2) can drive the reference plate (1) to move along the Z-axis direction, and the moving distance can be accurately read. 3. A geometric correction method of a texture image, adopting the result of the texture camera calibration method according to claim 1 to carry out image geometric correction, characterized in that: the correction process comprises the following steps: The first step is to generate a blank color image [ RM ₍ s , t ), G _M ( s , t ), B _M ( s , t )] to measure the camera’s viewing angle, and its components represent the pixels in the image ( s , t ) of red, green and blue primary color components; The second step, for each pixel coordinate ( s , t ) of the blank image generated in the first step, using the calibration result q _m of the texture camera calibration method according to claim 1, m=1,2,...,14, Substituting the three-dimensional coordinates [ X ( s , t ), Y ( s , t ), Z ( s , t )] of the object surface point corresponding to the pixel measured by the measurement system into the following formula, calculate its corresponding in the texture camera image pixel coordinates ( u , v ) The third step is to read the red, green and blue color components [ RT ( u , v ), G _T ( u , v ) , B _T ( u , v )] of the pixel ( u , v ₎ in the texture camera image; Fill the blank color image ( s , t ) pixels measuring the camera viewing angle with these color components, i.e. .