KR20090014927A - Method and apparatus for reproducing stereoscopic image using depth control - Google Patents

Abstract

A method and an apparatus for reproducing stereoscopic image are provided to control depths of objects and allow a computer to read a program for executing the method. Information about parallax at a left-eye image and a right-eye image of objects included in stereoscopic image is generated in accordance with stereo camera parameters of stereoscopic image. A depth at the stereoscopic image of the objects is controlled on the basis of the generated information. The stereoscopic image is reproduced in accordance with the controlled depth.

Description

Method and apparatus for reproducing stereoscopic image using depth control {Method and apparatus for reproducing stereoscopic image using depth control}

The present invention relates to a method and apparatus for reproducing a binocular image, and to a method and apparatus for reproducing a binocular image by minimizing eye fatigue felt by a viewer by controlling the depth of objects included in the binocular image. .

The biggest factor that humans feel in three dimensions is the spatial difference on the left and right retinas caused by looking at one object in different directions from the left and right eyes. In order to be able to feel such a difference in a display device that is a two-dimensional plane, a viewer may feel that the user is viewing a three-dimensional image by displaying different images, that is, a stereoscopic image, in the left and right eyes.

The viewer may make a 3D image feel by viewing the left eye image and the right eye image separately by wearing polarized glasses, or attach a lenticular screen to the display device without wearing glasses You can also display an image.

However, when a viewer watches such an artificial 3D image for a long time, eye fatigue may occur, and the viewer may cause severe vision deterioration and headache.

1A-1B show an example of a binocular image in accordance with the prior art.

1A to 1B illustrate a parallax between an image observed in a left eye camera and an image observed in a right eye camera when objects present in a world coordinate are observed from different left and right cameras. A computer graphics acceleration library such as DirectX is used to reproduce a three-dimensional object in the world coordinate system as a binocular image. The left eye camera 110 illustrated in FIG. 1A corresponds to the left eye of a viewer watching a binocular image, and the right eye camera 120 corresponds to the right eye of the viewer.

The image observed through the left eye camera is the same as the image 160 shown in the upper part of FIG. 1B, whereas the image observed through the right eye camera is the same as the image 170 shown in the lower part of FIG. 1B. Comparing the two images, the displacement occurs in the other objects (130 and 150) except for the rectangular object located in the center.

The displacements 170 and 180 of the objects 130 and 150 depend on the distance between the left eye camera 110 and the right eye camera 120 and the convergence angle formed by the optical axes of the two cameras. In general, the greater the distance between the two cameras 110 and 120, the greater the displacement.

Due to this displacement, a sense of depth of objects is generated, and a stereoscopic sense of a binocular image is formed from the sense of depth. However, if the displacement is too large, fatigue occurs due to the spatial heterogeneity that viewers feel in both eyes, so a method of reproducing the binocular image while minimizing the fatigue felt by the viewer by appropriately adjusting the depth of the objects of the binocular image is necessary. need.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide a method and apparatus for reproducing a binocular image by adjusting the depth of an object, wherein the computer-readable recording medium has recorded a program for executing the binocular image reproducing method. In providing.

In accordance with an aspect of the present invention, a method for reproducing a stereoscopic image according to a stereoscopic camera parameter of the binocular image may be used for disparity between left and right eye images of objects included in the binocular image. Generating information about the; Adjusting a depth in the binocular image of the objects based on the generated information; And reproducing the binocular image according to the adjusted depth.

According to a preferred embodiment of the present invention, the information on the parallax is characterized in that the information on the position difference in the left eye image and the right eye image of the objects.

According to a more preferred embodiment of the present invention, the generating of the information comprises: calculating a position in the left eye image of the objects according to the stereo camera parameter; Calculating a position in the right eye image of the objects according to the stereo camera parameter; And calculating a displacement of each of the objects based on a position in the left eye image and a position in the right eye image.

According to a more preferred embodiment according to the invention, the adjusting comprises adjusting the stereo camera parameter such that the magnitude of the displacement of each of the objects is equal to or less than a first threshold.

According to a more preferred embodiment according to the invention, the adjusting further comprises adjusting the stereo camera parameter such that the magnitude of displacement of some of the objects is less than or equal to a second threshold; Is smaller than the first threshold.

An apparatus for reproducing a stereoscopic image according to the present invention for solving the technical problem is based on the disparity between left and right eye images of objects included in the binocular image according to stereo camera parameters of the binocular image. Information generating unit for generating information about; A depth controller configured to adjust a depth in the binocular image of the objects based on the generated information; And a reproducing unit for reproducing the binocular image according to the adjusted depth.

According to a preferred embodiment of the present invention, the information generating unit calculates the position in the left eye image of the objects according to the stereo camera parameter, and calculates the position in the right eye image of the objects according to the stereo camera parameter And calculating displacements of the objects based on the position in the left eye image and the position in the right eye image.

According to a preferred embodiment of the present invention, the depth adjusting unit adjusts the stereo camera parameter so that the magnitude of the displacement of each of the objects is less than or equal to the first threshold.

According to a more preferred embodiment of the present invention, the depth adjusting unit adjusts the stereo camera parameter such that the displacement of some of the objects is less than or equal to a second threshold, and the second threshold is greater than the first threshold. It is characterized by a small one.

In order to solve the above technical problem, the present invention provides a computer-readable recording medium having recorded thereon a program for executing the method of reproducing the binocular image.

According to the present invention, the depth of the objects included in the binocular image may be automatically adjusted based on the displacement of the objects in the left eye image and the right eye image, thereby minimizing eye fatigue felt by the viewer, thereby reproducing the binocular image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 illustrates a binocular video reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the binocular image reproducing apparatus 200 according to an exemplary embodiment of the present invention includes an information generator 210, a depth controller 220, a reproducer 230, and a display device 240. do. 2 illustrates an apparatus for receiving a binocular image and playing it through a computer graphics acceleration library such as DirectX or OpenGL. The binocular image reproducing apparatus 200 receives information on objects and camera parameters included in the binocular image from a computer graphics acceleration library and reproduces the binocular image. The information about the objects may include information about the position of the objects in the world coordinate system, and the information about the camera parameters includes information about the camera parameters of the left eye camera and the offset between the left eye camera and the right eye camera. May be included.

The information generator 210 generates information about the parallax in the left eye image and the right eye image of the objects included in the binocular image. The position of the objects included in the binocular image is calculated from the left eye image and the right eye image using information about the position of the objects in the world coordinate system and camera parameters. Then generate information about the parallax based on the calculated position.

The parallax is a binocular parallax occurring in a binocular image, and information about parallax is generated by calculating a displacement of each object in a left eye image and a position of each object in a right eye image. The following equations explain how to calculate the displacement. When the position vector (4x1) in the world coordinate system of the nth object included in the binocular image is Xn and the position vector when projected to the left eye image is projected to xnl (3x1) and the right eye image If the vector is xnr (3 × 1), P1 is the left eye camera parameter (3 × 4), and P2 is the right eye camera parameter (3 × 4), xnl and xnr can be calculated by the following determinant.

xnl = P1 × Xn, xnr = P2 × Xn

Xn, the position of the nth object in the world coordinate system, is a matrix representing coordinates in three-dimensional space, and P1 and P2 are matrices representing parameters of left and right eye cameras. The right eye camera parameter P2 can be calculated from P1. In other words, the right eye camera parameter may be generated by applying information about a distance between the cameras and a convergence angle to the left eye camera parameter. Here, the convergence angle means an angle formed by the optical axis of the left eye camera and the optical axis of the right eye camera.

When the positions in the left eye image and the right eye image are determined for all objects included in the binocular image, the displacement dn may be calculated by the following equation (2).

dn = xnr-xnl

When the displacement is calculated for the objects included in the binocular image, a displacement histogram may be generated based on the calculation result. This will be described in detail with reference to FIG. 3.

3 shows a displacement histogram according to an embodiment of the present invention.

The information generator 210 may generate a histogram of the displacement of the objects in the left eye image and the right eye image as information about the parallax of the objects included in the binocular image. As described above with reference to FIGS. 1A and 1B, the displacement of each of the objects included in the binocular image is different depending on the position in the world coordinate system of each of the objects. Accordingly, a displacement histogram, such as the first histogram 350 of FIG. 3, may be generated according to the displacement of the objects.

Referring back to FIG. 2, the depth adjuster 320 adjusts depths of objects included in the binocular image based on information on parallax of objects generated by the information generator 310. The depth is adjusted by adjusting at least one of a distance and a convergence angle between the left eye camera and the right eye camera. Again, this will be described with reference to FIG. 3.

When the displacement histogram is generated as shown in the first histogram 350 of FIG. 3, the depth adjuster 320 may include the object such that the displacement of the objects of the binocular image is included in the maximum displacement 320 as shown in the second histogram 360. Adjust the depth Adjusting the depth of the objects so that the displacement of the objects fall within a predetermined range reduces eye fatigue felt by the viewer.

In other words, by adjusting the interval between the left eye camera and the right eye camera among the camera parameters, the magnitude of the displacement of all objects may be smaller than the magnitude of the displacement shown in the first histogram 350.

The depth adjuster 320 may adjust the camera parameter such that the displacement of the objects of the binocular image is included in the optimal displacement 310 as shown in the third histogram 370. As shown in the second histogram 360, the size of the displacement of all objects included in the binocular image is adjusted to be smaller overall, and then the displacements are shifted in the negative direction to include the displacements of all the objects in the optimum displacement 310. You can adjust the camera parameters so that

In particular, the displacements are shifted such that the displacement 330 of the object of interest is included in the optimum displacement 310. The object of interest is an object that the viewer focuses on in the binocular image, and the displacement 330 is mainly located on the right side of the displacement histograms 350, 360, and 370. As the object is closer to the viewer in the binocular image, the displacement is positive on the displacement histograms 350, 360, and 370, and thus is located on the right side in the displacement histograms 350, 360, and 370 of the object. Also, the closer the object is to the viewer, the higher the probability that the viewer is the object of interest that the viewer focuses on. Thus, the camera parameters are adjusted such that the displacement 330 of the object of interest is included in the optimum displacement 310.

The optimal displacement 310 is a displacement in which the eye fatigue felt by the viewer is minimized, and by adjusting the convergence angle between the left eye camera and the right eye camera, a displacement histogram in which the displacements are shifted in the negative direction as in the third histogram 370 is obtained. Can be.

The optimum displacement 310 and the maximum displacement 320 are numerical values that can be experimentally determined according to the size and type of the display device, and are not limited to specific values. For example, the optimal displacement 310 and the maximum displacement 320 may be determined based on the range of the optimal binocular disparity and the maximum binocular disparity of the 3D safety guideline of the 3D Consortium.

Referring back to FIG. 2, when the depth controller 220 adjusts the depth of the objects included in the binocular image by adjusting camera parameters, the playback unit 230 reproduces the binocular image according to the adjusted depth. The binocular image is rendered based on the adjusted camera parameters, that is, the distance and convergence angle between the left and right eye cameras. Fix left eye camera parameters and generate adjusted right eye camera parameters based on the adjusted spacing and convergence angle. Then, the binocular image is reproduced based on the left eye camera parameter and the adjusted right eye camera parameter.

The display device 240 receives the binocular image rendered by the playback unit 230 and displays the image to the viewer.

4 is a flowchart illustrating a method of reproducing a binocular image according to an embodiment of the present invention.

Referring to FIG. 4, in operation 410, the binocular image reproducing apparatus 200 generates information about a parallax in a left eye image and a right eye image of objects included in the binocular image. The position of the objects in the left eye image and the position of the right eye image are calculated using camera parameters, and information on parallax is generated by calculating displacement based on the calculation result.

In operation 420, the binocular image reproducing apparatus 200 adjusts the depths of the objects based on the information on the parallax generated in operation 410. The depth of the objects is adjusted by adjusting camera parameters based on information about the displacement of objects in the left eye image and the right eye image. The depth of the objects is adjusted by adjusting at least one of a distance and a convergence angle between the left eye camera and the right eye camera among the camera parameters.

In operation 430, the binocular image reproducing apparatus 200 calculates displacements of the objects according to the depths of the objects in operation 420, and determines whether the calculated displacement is less than or equal to the first threshold. Here, the first threshold may be the maximum displacement 320 described above with reference to FIG. 3.

If it is determined that the magnitude of the displacement of the objects is not smaller than the first threshold, the depth of the objects is adjusted again in step 420. If it is determined that the magnitude of the displacement of all objects is smaller than the first threshold, it is determined in step 440 whether the magnitude of the displacement of the object of interest is less than or equal to the second threshold. Here, the second threshold is a value smaller than the first threshold, and may preferably be the optimum displacement 310 described above with reference to FIG. 3.

If it is determined in step 440 that the displacement of the object of interest is less than or equal to the second threshold, in step 450 the binocular image reproducing apparatus 200 reproduces the binocular image according to the adjusted depth. The binocular image is reproduced according to the adjusted camera parameter, that is, the distance and convergence angle between the left eye camera and the right eye camera.

5 illustrates a user interface according to an embodiment of the present invention.

5 illustrates a user interface displaying a depth adjustment mode provided to a viewer through a display device for reproducing a binocular image according to the binocular image reproducing method according to the present invention.

According to the related art, in order to adjust the depth of the binocular image, the viewer has to adjust the depth of the objects directly through the manual adjustment mode 530. However, according to the present invention, an automatic adjustment mode for generating displacement histograms 350, 360, and 370 of objects included in the binocular image based on the parallax information, and automatically adjusting the depth in accordance with the generated displacement histogram. A user interface is provided for the viewer to select 520.

When the viewer selects the automatic adjustment mode 520 in the provided user interface, the binocular image is reproduced by automatically adjusting the depth in accordance with the binocular image reproduction method according to the present invention.

As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications will fall within the scope of the invention. In addition, the system according to the present invention can be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and also include a carrier wave (for example, transmission through the Internet). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

1A-1B show an example of a binocular image in accordance with the prior art.

2 illustrates a binocular video reproducing apparatus according to an embodiment of the present invention.

3 shows a displacement histogram according to an embodiment of the present invention.

4 is a flowchart illustrating a method of reproducing a binocular image according to an embodiment of the present invention.

5 illustrates a user interface according to an embodiment of the present invention.

Claims (17)

In the method of reproducing a stereoscopic image, Generating information about a parallax in a left eye image and a right eye image of objects included in the binocular image according to a stereo camera parameter of the binocular image; Adjusting a depth in the binocular image of the objects based on the generated information; And And reproducing the binocular image according to the adjusted depth. The method of claim 1, wherein the time difference information Binocular image playback method characterized in that the information on the position difference between the left eye image and the right eye image of the objects. The method of claim 2, wherein generating the information Calculating a position in the left eye image of the objects according to the stereo camera parameter; Calculating a position in the right eye image of the objects according to the stereo camera parameter; And And calculating a displacement of each of the objects based on the position in the left eye image and the position in the right eye image. 4. The method of claim 3, wherein said adjusting is And adjusting the stereo camera parameter such that the magnitude of the displacement of each of the objects is less than or equal to a first threshold. 5. The method of claim 4, wherein said adjusting is Adjusting the stereo camera parameter such that the magnitude of displacement of some of the objects is less than or equal to a second threshold; And the second threshold is smaller than the first threshold. 5. The method of claim 4, wherein said adjusting is And adjusting a distance between the camera for the left eye image and the camera for the right eye image. The method of claim 6, wherein the reproducing step And reproducing the binocular image according to the camera parameter for the right eye image adjusted according to the camera parameter for the left eye image and the adjusted camera. Way. 5. The method of claim 4, wherein said adjusting is And adjusting the convergence angle of the camera for the left eye image and the camera for the right eye image. In the device for reproducing a stereoscopic image, An information generation unit for generating information on parallax in left and right eye images of objects included in the binocular image according to stereo camera parameters of the binocular image; A depth controller configured to adjust a depth in the binocular image of the objects based on the generated information; And And a reproduction unit for reproducing the binocular image according to the adjusted depth. The method of claim 9, wherein the time difference information The binocular image reproducing apparatus, characterized in that the information on the position difference in the left eye image and right eye image of the objects. The method of claim 10, wherein the information generating unit The position in the left eye image of the objects is calculated according to the stereo camera parameter, and the position in the right eye image of the objects is calculated according to the stereo camera parameter, and the position in the left eye image and the position in the right eye image are calculated. And a displacement of each of the objects based on the two-eye image reproducing apparatus. The method of claim 11, wherein the depth adjustment unit And adjusting the stereo camera parameter such that the displacement of each of the objects is equal to or less than a first threshold. The method of claim 12, wherein the depth adjustment unit Adjust the stereo camera parameter such that the magnitude of displacement of some of the objects is less than or equal to a second threshold, And the second threshold is smaller than the first threshold. The method of claim 11, wherein the depth adjustment unit And a distance between the camera for the left eye image and the camera for the right eye image. The method of claim 14, wherein the regeneration unit The binocular image reproducing apparatus, characterized in that for playing the binocular image in accordance with the parameter of the camera for the left eye image and the parameter of the camera for the right eye image adjusted according to the interval between the adjusted camera. The method of claim 11, wherein the depth adjustment unit And a converging angle between the camera for the left eye image and the camera for the right eye image. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 8.

Images