WO2012092758A1

WO2012092758A1 - Method and device for correcting translation error of images picked up by two cameras

Info

Publication number: WO2012092758A1
Application number: PCT/CN2011/076160
Authority: WO
Inventors: 李春雨; 张薇; 李志远
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-01-05
Filing date: 2011-06-22
Publication date: 2012-07-12
Also published as: CN102123290A; CN102123290B

Abstract

A method for correcting translation error of images picked up by two cameras is disclosed by the present invention. The method includes: picking up calibration objects by using the two cameras simultaneously to obtain a left eye calibration image and a right eye calibration image respectively (401); determining the translation error between the two cameras according to the positions of the calibration objects in the two calibration images and saving the translation error (402); and correcting the images picked up by the two cameras according to the saved translation error (403). A device for correcting translation error of images picked up by the two cameras is also disclosed by the present invention. According to the method and device, the translation error between the two cameras can be eliminated effectively, and viewers can watch clear stereo images.

Description

Method and device for correcting translation error of image captured by dual camera

The present invention relates to a dual camera stereoscopic imaging technique, and more particularly to a method and apparatus for correcting translational errors of images captured by dual cameras. Background technique

In the prior art, the stereoscopic display of pictures or images is realized by the parallax characteristics of both eyes, and the main methods are: color division method, photo division method, time division method and grating method. Although the specific implementation methods are different, the basic starting point is the same, and the principle is similar. That is, using two cameras placed side by side to represent the left and right eyes of the person respectively, two cameras are used to simultaneously capture two horizontal parallaxes. Picture. During the screening, the left-eye film and the right-eye film are respectively loaded into the left-eye projection device and the right-eye projection device, and the left-eye projection device and the right-eye projection device are synchronized to display the screen, and a double image including the left-eye image and the right-eye image is formed. Shadow image. Through some special equipment, such as polarized glasses, the viewer can only see the left eye image of the viewer's left eye, while the right eye can only see the right eye image. Then, through the convergence function of the eyes of the audience, the left eye image and the right eye image are respectively superimposed on the retina film, and the three-dimensional visual effect is generated by the brain nerve.

The above four stereo display and shooting techniques require the use of dual cameras for shooting,

1(a) and Fig. 1(b) are schematic diagrams of the left-eye image and the right-eye image when the dual camera is properly installed, as shown in Fig. 1(a) and Fig. 1(b), the two images are left-eye cameras and The left eye image and the right eye image taken by the right eye camera. However, there may be errors in the position of the dual camera when the dual camera is mounted on the camera, or the vibration of the camera may cause errors in the camera's dual camera position. For example: Dual camera has left and right, up and down positions Offset, etc. When one of the two cameras, such as the right-eye camera, has a left-right offset, the captured image will shift. Figures 2(a) and 2(b) show the dual-camera position when the right-eye camera is offset. Captured image The schematic diagram, as shown in Figures 2(a) and 2(b), shifts the right eye image slightly to the left due to the offset of the right eye camera. Figure 3 is a schematic diagram of superimposition of the left-eye image and the right-eye image when the right-eye camera is shifted. As shown in Figure 3, the superimposed image appears offset so that the viewer perceives that the edges of the image are blurred or even ghosted.

Currently, the dual camera unit performs calibration of the camera before leaving the factory to correct problems such as radial distortion of the camera, so that the captured image is not significantly distorted. Commonly used are double plane calibration methods, Tsai two-step method, self-calibration method, and so on. However, these calibration techniques generally calibrate a single camera itself, rather than the cameras. Therefore, the mounting error between the dual cameras cannot be completely eliminated. Summary of the invention

In view of the above, it is a primary object of the present invention to provide a method and apparatus for correcting a translation error of an image captured by a dual camera, which can completely eliminate the positional error between the dual cameras, resulting in a problem of offset of the superimposed image.

In order to achieve the above object, the technical solution of the present invention is achieved as follows:

The present invention provides a method of correcting a translation error of an image captured by a dual camera, the method comprising:

Using a dual camera to simultaneously capture the calibration object, respectively obtaining a left eye calibration image and a right eye calibration image;

Determining a translation error between the dual cameras based on the position of the calibration object in the two calibration images, and saving the translation error;

Corrects the image captured by the dual camera based on the saved translation error.

Wherein, the calibration object is: four calibration objects in the same plane, the plane is perpendicular to the ground and parallel to the connection of the optical center of the dual camera, and the horizontal coordinate spacing of the first calibration object and the second calibration object is equal to the dual camera light The spacing of the hearts, and the abscissa spacing of the third and fourth calibration objects is also equal to the spacing of the optical centers of the dual cameras. Wherein, determining the translation error between the dual cameras is: respectively determining a longitudinal error and a lateral error of the calibration object in the left eye calibration image and the right eye calibration image.

Wherein, the longitudinal error is obtained by calculating a difference between an average value of a left eye ordinate of the first calibration object and the third calibration object, and a difference between an average value of the right eye ordinate of the second calibration object and the fourth calibration object; Calculating the difference between the average value of the left eye abscissa of the first calibration object and the third calibration object, and the difference between the average value of the right eye and the right eye of the fourth calibration object.

Wherein, the correcting the image captured by the dual camera is: performing longitudinal, and/or lateral translation on the left eye image, and/or the right eye image according to the translation error.

The present invention also provides an apparatus for correcting a translation error of an image captured by a dual camera, the apparatus comprising: a calibration image collection module, a translation error determination module, and an image correction module, wherein

The calibration image collection module is configured to simultaneously capture a calibration object by using a dual camera, respectively obtain a left eye calibration image and a right eye calibration image, and send the two calibration images to the translation error determination module;

The translation error determining module is configured to determine a translation error between the dual cameras according to the position of the calibration object in the two calibration images, and save the translation error;

The image correction module is configured to correct the image captured by the dual camera according to the saved translation error.

Wherein, the calibration object simultaneously captured by the dual camera of the calibration image collection module is: four calibration objects in the same plane, the plane is perpendicular to the ground and parallel to the connection of the optical center of the dual camera, the first calibration object The abscissa spacing of the second calibration object is equal to the spacing of the optical centers of the dual cameras, and the abscissa spacing of the third calibration object and the fourth calibration object is also equal to the spacing of the optical centers of the dual cameras.

The method and device for correcting the translation error of the image captured by the dual camera provided by the invention, using the dual camera to simultaneously capture the calibration object, respectively obtaining the left eye calibration image and the right eye calibration image; The translation error between the dual cameras is determined according to the position of the calibration object in the two calibration images, and the translation error is saved; the image captured by the dual camera is corrected according to the saved translation error. It can effectively eliminate the translation error between the two cameras, so that the viewer can see the clear stereo image. Further, if the camera is bumped during use, causing the dual camera to be offset, the method and the device of the present invention can be recalibrated by the user, and the manufacturer does not need to be sent back for maintenance, which facilitates the user's later use. Maintenance in the middle. DRAWINGS

Figure 1 (a) and Figure 1 (b) are schematic diagrams of the left eye image and the right eye image when the dual camera is properly installed; Figure 2 (a) and Figure 2 (b) are taken by the dual camera when the right eye camera is offset. Schematic diagram of the image;

3 is a schematic diagram of superimposition of a left eye image and a right eye image when a right eye camera position is shifted; FIG. 4 is a schematic flow chart of a method for correcting a translation error of an image captured by a dual camera according to the present invention;

FIG. 5 is a schematic diagram of an effect of calibrating an image of a left eye according to an embodiment of the present invention; FIG.

6 is a schematic diagram of an effect of calibrating an image of a right eye according to an embodiment of the present invention;

Fig. 7 is a schematic view showing the structure of a device for correcting a translation error of an image captured by a dual camera. detailed description

In the prior art, the camera uses a lens as a lens for imaging purposes. When light in all directions passes through the lens, it will deflect and change the direction of light propagation. However, there is a point on the lens. When the light passing through any direction passes through the point, the direction of propagation of the light is constant, that is, the direction of the exit and the direction of the incident are parallel to each other. This is called the optical center of the lens, referred to as the optical center. In the present invention, it is determined that the calibration position of the camera is based on the optical center of the lens.

The basic idea of the invention is: using a dual camera to simultaneously capture the calibration object, respectively, to get left The eye calibration image and the right eye calibration image; determining a translation error between the dual cameras according to the position of the calibration object in the two calibration images, and saving the translation error; correcting the image captured by the dual camera according to the saved translation error.

The technical solutions of the present invention are further elaborated below in conjunction with the accompanying drawings and specific embodiments. 4 is a schematic flow chart of a method for correcting a translation error of an image captured by a dual camera according to the present invention. As shown in FIG. 4, the method for correcting a translation error of an image captured by a dual camera includes the following steps:

Step 401, using a dual camera to simultaneously capture a calibration object, and obtaining a left eye calibration image and a right eye calibration image, respectively;

Specifically, the calibration object is in the same plane, the plane is perpendicular to the ground and parallel to the line connecting the optical centers of the dual cameras, and the abscissa distance between the first calibration object and the second calibration object is the same as the third calibration object and the fourth The calibration object is equal to the pitch of the optical center of the dual camera.

Step 402: Determine a translation error between the dual cameras according to the position of the calibration object in the two calibration images, and save the translation error;

Specifically, the determining a translation error between the dual cameras is specifically: determining a longitudinal error and a lateral error of the calibration object in the left eye calibration image and the right eye calibration image, respectively. The specific implementation manner is: calculating the average value of the left eye ordinate of the first calibration object and the third calibration object, and the difference between the second calibration object and the average value of the right eye ordinate of the fourth calibration object as a longitudinal error; calculating the first calibration The difference between the average of the left eye abscissa of the object and the third calibration object, and the difference between the average value of the right eye and the right eye of the fourth calibration object as the lateral error.

Step 403: Correct the image captured by the dual camera according to the saved translation error.

Specifically, the correcting the image captured by the dual camera is specifically performing longitudinal, and/or lateral translation on the left eye image and/or the right eye image according to the translation error to correct the translation error of the image. In the prior art, the digital image processing technology is very mature, so that the digital image can be fully translated, twisted, stretched, scaled, and cropped. Deformation transformation of digital images, such as compression Transform or stretch transformation is the coordinate transformation of the original image to the target image. Simply put, the coordinates of each point of the original image are converted into the new coordinates of the corresponding points of the target image by the deformation operation. Therefore, it is completely possible to perform compression conversion processing or stretch transformation processing on a digital image. The digital image segmentation technique is based on the coordinates of pixel points, and the digital image is formed into a new digital image according to certain rules. The merging technique of digital images is based on the coordinates of pixel points, and two or more digital images are combined according to certain rules to form a new digital image. The image to be corrected can be easily translated by the above digital image processing technology, and the corrected image can be further optimized by a technique such as segmentation.

The method of the present invention will now be described in detail in connection with a specific embodiment.

FIG. 5 is a schematic diagram showing the effect of the calibration image of the left eye in the embodiment of the present invention. As shown in FIG. 5, the left-eye calibration image has an abscissa of 31 and an ordinate of 32, and the first calibration object 1 has coordinates (xl l , yl l ), the second calibration object 2 coordinates are (xl2, yl2), the third calibration object 3 coordinates are (xl3, yl3), and the fourth calibration object 4 coordinates are (xl4, yl4). 6 is a schematic diagram showing the effect of the right eye calibration image in the embodiment of the present invention. As shown in FIG. 6, the right eye calibration image has an abscissa of 41 and an ordinate of 42, and the first calibration object 1 has coordinates (x21, y21), The coordinates of the second calibration object 2 are (x22, y22), the coordinates of the third calibration object 3 are (x23, y23), and the coordinates of the fourth calibration object 4 are (x24, y24). Meanwhile, Xl2-xl l = xl4 - xl3 = x22 - x21 = x24 - x23 is equal to the optical distance between the left camera 15 and the right camera 16.

Assuming that the dual camera is installed correctly, that is, there is no offset problem, the four calibration objects are in the same coordinate system of the two calibration images, and the ordinate values are the same, namely: yl l=y21; yl2=y22; yl3=y23 ; yl4=y24.

Since the distance between the first calibration object 1 and the second calibration object 2 on the left-eye abscissa 31 is equal to the distance between the respective optical centers of the dual cameras, in the left-eye calibration image and the right-eye calibration image, the first calibration object 1 and the second The calibration object 2 has the same abscissa in the coordinate system of each of the two calibration images, that is, xl l=x22. Since the distance between the third calibration object 3 and the fourth calibration object 4 on the right eye abscissa 41 is equal to the distance between the respective optical centers of the dual cameras, the left eye calibration image and the right eye calibration image, the third calibration object 3, the fourth The calibration object 4 has the same abscissa in the coordinate system of each of the two pictures, that is, xl3=x24.

Assume that when the right camera is installed, the error of 2 pixels is shifted to the right, and the coordinates of the calibration object captured by the left camera in the left-eye calibration image are as follows:

The first calibration object 1, the coordinates are (xll, yll), ie (75, 2000),

The second calibration object 2, the coordinates are (xl2, yl2), ie (140, 1800),

The third calibration object 3, the coordinates are (xl3, yl3), ie (1075, 210),

The fourth calibration object 4, the coordinates are (xl4, yl4), ie (1140, 10).

The coordinates of the calibration object captured by the right camera in the calibration image of the right eye are as follows:

The first calibration object 1, the coordinates are (x21, y21), ie (8, 2000),

The second calibration object 2, the coordinates are (x22, y22), ie (73, 1800),

The third calibration object 3, the coordinates are (x23, y23), ie (1008, 210),

The fourth calibration object 4, the coordinates are (x24, y24), ie (1073, 10).

Calculating the left-eye ordinate average Y_Nol of the first calibration object 1 and the third calibration object 3 on the left-eye ordinate 32, and calculating the second calibration object 2 and the fourth calibration object 4 on the right-eye ordinate 42 Right eye ordinate mean Y—No2;

Y Nol = (yll+yl3 ) 12= (2000+210) /2= 1105

Y_No2 = ( y21+ y23 ) /2 = ( 2000+210 ) 12 = 1105

Calculating the left-eye horizontal coordinate X_Nol of the first calibration object 1 and the third calibration object 3 on the left-eye abscissa 31, and calculating the second calibration object 2 and the fourth calibration object 4 on the right-eye abscissa 41 Right eye horizontal coordinate average X—No2;

X— Nol = ( xll+xl3 ) /2= ( 75+1075 ) 12 = 575

X No2 = ( x21+x23 ) 12= { 73+1073 ) 12 = 573 The average value of the ordinate of the right eye is subtracted from the average value of the ordinate of the left eye, and the obtained ordinate difference DY is the longitudinal error, and the average value of the right eye abscissa is subtracted from the left eye horizontal coordinate average, and the obtained horizontal coordinate difference is obtained. The value DX is the lateral error. At the same time, the translation error is saved for image correction when the image is taken.

DY = Y_No2 - Y Nol = 1105 - 1105 = 0

DX = X_No2 - X Nol = 575 - 573 = 2

The right eye image is translated along the positive direction of the ordinate, and the ordinate difference is DY pixel units upward; the right eye image is translated along the positive direction of the horizontal coordinate, and the horizontal coordinate difference is DX pixel units to the right, that is, the right eye image is upward. Panning 0 pixels; Shifting the right eye image to the right by 2 pixels along the abscissa, then correcting the image taken by the dual camera according to the saved translation error.

7 is a schematic structural diagram of an apparatus for correcting a translation error of an image captured by a dual camera according to the present invention. As shown in FIG. 7, the apparatus for correcting a translation error of an image captured by a dual camera includes: a calibration image collection module 71, and a translation The error determining module 72 and the image correcting module 73, wherein the calibration image collecting module 71 is configured to simultaneously capture a calibration object by using a dual camera, and obtain a left eye calibration image and a right eye calibration image, respectively, and two calibration images. Send to the translation error determination module 72;

The translation error determination module 72 is configured to determine a translation error between the dual cameras according to the position of the calibration object in the two calibration images, and save the translation error;

Specifically, the translation error determination module 72 determines a translation error between the dual cameras, specifically: determining longitudinal and lateral errors of the calibration object in the left-eye calibration image and the right-eye calibration image, respectively. The specific implementation manner is: calculating the average value of the left eye ordinate of the first calibration object and the third calibration object, and the difference between the second calibration object and the average value of the right eye ordinate of the fourth calibration object as a longitudinal error; calculating the first calibration The difference between the average of the left eye abscissa of the object and the third calibration object, and the difference between the average value of the right eye and the right eye of the fourth calibration object as the lateral error. The image correction module 73 is configured to correct an image captured by the dual camera according to the saved translation error.

Specifically, the image correction module 73 corrects the image captured by the dual camera, specifically performing longitudinal, and/or lateral translation on the left eye image and/or the right eye image according to the translation error to correct the translation error of the image. In the prior art, digital image processing technology is very mature, so that digital images can be panned, twisted, stretched, scaled, and cropped. The deformation transformation of the digital image, such as compression transformation or stretch transformation, is the coordinate transformation of the original image to the target image. Simply put, the coordinates of each point of the original image are converted into the new coordinates of the corresponding point of the target image by the deformation operation. . Therefore, it is completely possible to perform compression conversion processing or stretch transformation processing on a digital image. The digital image segmentation technique is based on the coordinates of pixel points, and the digital image is formed into a new digital image according to certain rules. The merging technique of digital images is based on the coordinates of pixel points, and two or more digital images are combined according to certain rules to form a new digital image. The image to be corrected can be easily translated by the above digital image processing technology, and the corrected image can be further optimized by a technique such as segmentation.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included. Within the scope of protection of the present invention.

Claims

Claim

A method for correcting a translation error of an image captured by a dual camera, the method comprising:

2. The method according to claim 1, wherein the calibration object is: four calibration objects in the same plane, the plane is perpendicular to the ground and parallel to the connection of the optical center of the dual camera, the first calibration object The abscissa spacing of the second calibration object is equal to the spacing of the optical centers of the dual cameras, and the abscissa spacing of the third calibration object and the fourth calibration object is equal to the spacing of the optical centers of the dual cameras.

3. The method according to claim 2, wherein the determining a translation error between the dual cameras is: determining a longitudinal error and a lateral error of the calibration object in the left eye calibration image and the right eye calibration image, respectively.

4. The method according to claim 3, wherein the longitudinal error is calculated by calculating a mean value of a left eye ordinate of the first calibration object and the third calibration object, and a right calibration of the second calibration object and the fourth calibration object The difference between the mean values of the coordinates is obtained;

The lateral error is obtained by calculating a difference between the left-axis abscissa mean value of the first calibration object and the third calibration object, and the difference between the average value of the right-eye abscissa of the second calibration object and the fourth calibration object.

The method according to claim 1 or 2, wherein the correcting the image captured by the dual camera is: performing longitudinal, and/or lateral translation of the left eye image, and/or the right eye image according to the translation error.

6. A device for correcting a translation error of an image captured by a dual camera, the device comprising: a calibration image collection module, a translation error determination module, and an image correction module, Medium,

The apparatus according to claim 6, wherein the calibration object simultaneously captured by the dual camera of the calibration image collection module is: four calibration objects in the same plane, the plane is perpendicular to the ground and parallel In the connection of the optical center of the dual camera, the abscissa distance between the first calibration object and the second calibration object is equal to the distance between the optical centers of the dual cameras, and the abscissa distance between the third calibration object and the fourth calibration object is also equal to the dual camera optical center. Pitch.

8. The apparatus according to claim 7, wherein the translation error determination module determines that a translation error between the dual cameras is: respectively determining a longitudinal error of the calibration object in the left-eye calibration image and the right-eye calibration image. Lateral error.

9. The apparatus according to claim 8, wherein the translation error determination module determines that the longitudinal error is obtained by calculating the average of the left eye ordinate of the first calibration object and the third calibration object, and the second calibration object and the The difference between the average values of the right eye ordinates of the four calibration objects is obtained;

The translation error determination module determines that the lateral error is obtained by calculating a difference between the average value of the left eye abscissa of the first calibration object and the third calibration object, and the average value of the right eye abscissa of the second calibration object and the fourth calibration object.

10. The apparatus according to claim 6 or 7, wherein the image correcting module corrects the image captured by the dual camera is: performing a slanting direction on the left eye image, and/or the right eye image according to the translation error / or lateral translation.