KR101198557B1 - 3D stereoscopic image and video that is responsive to viewing angle and position - Google Patents

Abstract

The present invention relates to a system and method for generating a three-dimensional stereoscopic image that has a function of changing the stereoscopic image according to the tilting and position of the viewer's head and actively reacts to the changing of the viewing posture.
The three-dimensional stereoscopic image generation system according to the present invention includes an image acquisition unit for acquiring the main image and the auxiliary image according to the type of the input image, and a viewer posture collecting unit for collecting the viewing attitude information about the viewing direction and the viewing position of the viewer. It characterized in that it comprises a three-dimensional stereoscopic image generating unit for generating a three-dimensional stereoscopic image in accordance with the viewer's attitude from one image.

Description

3D stereoscopic image and video that is responsive to viewing angle and position}

The present invention relates to a method for generating and playing 3D stereoscopic images, and more particularly, to 3D stereoscopic images which actively respond to changes in viewing posture, having a function of changing stereoscopic images according to the tilt and position of a viewer's head. A system and method for generating images.

With the development of information devices, technology for authoring and utilizing 3D stereoscopic displays and digital contents using them in the general consumer electronics market and entertainment market is becoming important, providing a more realistic and less dizzy experience of 3D stereoscopic images. The technique to do is researched. For visualization of 3D stereoscopic images, stereoscopic recognition method using binocular disparity based on spatiotemporal cognitive model of human eyes and brain is most widely used.

However, in general stereoscopic image generation techniques, the display device is fixed, and the viewer's attitude in which the viewer's eyes look horizontally at the display device is recommended. This is because a general stereoscopic image generation technique uses a stereoscopic recognition method using binocular disparity, and it is assumed that left and right eye images exposed to both eyes are horizontally arranged. Therefore, the conventional stereoscopic image generation technique can accurately generate stereoscopic images only when viewing horizontal images through horizontal photographing and horizontal playback, and attempts to watch the viewer by moving the head without tilting the head or looking at the screen from the front. May not respond appropriately. For example, when a viewer views a stereoscopic image generated by horizontal photographing and horizontal playback by tilting the head by 90 degrees, the displacement direction of the image that generates binocular disparity and the actual viewer's displacement direction cross 90 degrees. May not be appropriately recognized.

KR 10-0381817 B1 KR 10-0538471 B1

One problem to be solved by the present invention is a method for generating a three-dimensional stereoscopic image that changes according to the viewer's head tilt in response to the viewer's attitude, and a three-dimensional stereoscopic image including a beam projector and a theater-type large-screen display including a 3D TV. It is to provide a system and method of creation.

In addition, another problem to be solved by the present invention is a method for generating a three-dimensional stereoscopic image that changes according to the viewer's head tilt, movement, and position in response to the viewer's attitude, and a beam projector and a theater-type large screen including a 3D TV. The present invention provides a system and method for generating a 3D stereoscopic image including a display.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

One aspect of the three-dimensional stereoscopic image generating system of the present invention for achieving the above object is an input image unit for receiving a variety of input images; An image acquiring unit performing a preparation step for acquiring an image and generating a 3D stereoscopic image; A viewer unit for providing basic information on the viewer's attitude; A viewer posture collecting unit for collecting information on a viewing direction and a position from the viewer unit; A three-dimensional stereoscopic image generation unit for converting an image provided from the image acquisition unit into an appropriate three-dimensional stereoscopic image according to the viewing direction and viewing position information; It includes a stereoscopic display unit for showing the generated three-dimensional stereoscopic image to the viewer.

Another aspect of the three-dimensional stereoscopic image generating system of the present invention to achieve the above object is an input image unit for receiving a variety of input images; A viewer unit for providing basic information on the viewer's attitude; A viewer posture collecting unit for collecting information on a viewing direction and a position from the viewer unit; A three-dimensional stereoscopic image generation unit for converting an image provided from the image acquisition unit into an appropriate three-dimensional stereoscopic image according to the viewing direction and viewing position information; It includes a stereoscopic display unit for showing the generated three-dimensional stereoscopic image to the viewer.

Another aspect of the three-dimensional stereoscopic image generation system of the present invention to achieve the above object is a viewer unit for providing basic information about the viewer's viewing position; A viewer posture collecting unit for collecting information on a viewing direction and a position from the viewer unit; A three-dimensional stereoscopic image generation unit for converting an image provided from the image acquisition unit into an appropriate three-dimensional stereoscopic image according to the viewing direction and viewing position information; It includes a stereoscopic display unit for showing the generated three-dimensional stereoscopic image to the viewer.

Another aspect of the three-dimensional stereoscopic image generating system of the present invention for achieving the above object is an input image unit for receiving a variety of input images; An image acquiring unit performing a preparation step for acquiring an image and generating a 3D stereoscopic image; A viewer unit for providing basic information on the viewer's attitude; A viewer posture collecting unit for collecting information on the viewing direction from the viewer unit; A three-dimensional stereoscopic image generation unit for converting an image provided from the image acquisition unit into an appropriate three-dimensional stereoscopic image according to a viewing direction; It includes a stereoscopic display unit for showing the generated three-dimensional stereoscopic image to the viewer.

Another aspect of the three-dimensional stereoscopic image generating system of the present invention for achieving the above object is an input image unit for receiving a variety of input images; A viewer unit for providing basic information on the viewer's attitude; A viewer posture collecting unit for collecting information on the viewing direction from the viewer unit; A three-dimensional stereoscopic image generation unit for converting an image provided from the image acquisition unit into an appropriate three-dimensional stereoscopic image according to a viewing direction; It includes a stereoscopic display unit for showing the generated three-dimensional stereoscopic image to the viewer.

Another aspect of the three-dimensional stereoscopic image generation system of the present invention to achieve the above object is a viewer unit for providing basic information about the viewer's viewing position; A viewer posture collecting unit for collecting information on the viewing direction from the viewer unit; A three-dimensional stereoscopic image generation unit for converting an image provided from the image acquisition unit into an appropriate three-dimensional stereoscopic image according to a viewing direction; It includes a stereoscopic display unit for showing the generated three-dimensional stereoscopic image to the viewer.

An example of the viewing posture described in the present invention refers to a viewing direction including tilting of a head of a viewer watching a 3D stereoscopic image. Another example of the viewing posture described in the present invention refers to a viewing position including the positions of both eyes of a viewer watching a 3D stereoscopic image. Another example of the viewing posture described in the present invention refers to the viewing position including the viewing direction and the positions of both eyes including the head tilt of the viewer watching the 3D stereoscopic image. The viewing posture of the present invention is not limited to the viewing directions and viewing positions mentioned above, and other viewing posture information not mentioned will be clearly understood by those skilled in the art from the following description.

Specific details of other embodiments are included in the detailed description and the drawings.

Using the 3D stereoscopic image generating system and method according to the present invention, the viewing direction and the viewing position information of the viewer are collected to calculate the direction and position of the viewpoint used for generating the left and right eye images constituting the 3D stereoscopic image. And can be reflected in the generation of 3D stereoscopic images. Since the 3D stereoscopic image is appropriately responded to the position of the head without the viewer tilting the head or looking at the front of the display, the direction of the displacement difference between the left and right eye images of the 3D stereoscopic image according to the present invention is It can always be kept in the same direction as the viewer's binocular. In addition, since the left eye image and the right eye image correspond appropriately according to the change in the direction and position of the viewer's head, three-dimensional stereoscopic recognition due to the parallax of both eyes and stereo parallax effect due to the motion parallax according to the movement of the viewpoint. Can be obtained.

Therefore, the 3D stereoscopic image generating system according to the present invention can provide a 3D stereoscopic experience with higher realism and less dizziness than a general stereoscopic image generating method.

1 is a block diagram of a 3D stereoscopic image generating system reflecting a viewing attitude according to an exemplary embodiment of the present invention.
2 is a view illustrating an input image unit 200, an image acquisition unit 300, a viewer unit 400, and a viewer posture collection unit 500 of a 3D stereoscopic image generation system reflecting a viewer's attitude according to an embodiment of the present invention. And a block diagram showing the flow of the 3D stereoscopic image generator 600.
3 is a view showing an example of an image acquisition process for authoring a three-dimensional stereoscopic image 240 of an embodiment of the present invention.
4 is a view showing an example of three-dimensional glasses for collecting the viewing attitude information 410 of one embodiment of the present invention.
FIG. 5 is a diagram illustrating an example of a method of calculating a left eye image 620 and a right eye image 630 generated by the 3D stereoscopic image generator 600 according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating another example of a method of calculating a left eye image 620 and a right eye image 630 generated by the 3D stereoscopic image generator 600 according to an embodiment of the present invention.
FIG. 7 illustrates a left eye image 620 and a right eye image 630 generated by the 3D stereoscopic image generator by reflecting a viewer's attitude viewed horizontally from the front of the stereoscopic display unit 700 according to an exemplary embodiment of the present invention. This is a view showing an example of the three-dimensional display unit 700 is visualized.
8 is a left eye image 620 generated by the 3D stereoscopic image generating unit by reflecting the viewer's attitude in which the viewer tilts his head 45 degrees to the right from the front of the stereoscopic display unit 700 and the viewer of the exemplary embodiment of the present invention; The right eye image 630 is a view showing another example of the three-dimensional display unit 700 is visible.
FIG. 9 illustrates a left eye image 620 generated by the 3D stereoscopic image generator by reflecting a viewer's attitude viewed vertically by turning the head 90 degrees to the right from the front of the stereoscopic display unit 700 according to an embodiment of the present invention. ) And the right eye image 630 are views illustrating another example of the stereoscopic display unit 700.

Advantages and features of the present invention, and methods of achieving the same will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments set forth below, but may be implemented in various forms, and only the present embodiments make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

Hereinafter, the present invention will be described with reference to the drawings of a block diagram for explaining a 3D stereoscopic image generating system and method according to embodiments of the present invention. At this time, it will be appreciated that combinations of each block in the block diagram may be performed by computer program instructions. These computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, such that the instructions executed by the processor of the computer or other programmable data processing equipment may be executed in block (s) of the block diagram. It will create means for performing the described functions. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in the block (s). Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for executing the functions described in block (s).

In addition, each block may represent a portion of a module, segment, or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, the two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the corresponding function.

The term '~ part' used in the present embodiment refers to software or hardware or a content component, and '~ part' plays certain roles. However, '~' is not meant to be limited to software, hardware, or content. &Quot; to " may be configured to reside on an addressable storage medium and may be configured to play one or more processors. Thus, as an example, '~' means components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, and the like. And subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and the 'parts' may be combined into a smaller number of components and the 'parts' or further separated into additional components and the 'parts'.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a 3D stereoscopic image generating system reflecting a viewer's attitude according to an embodiment of the present invention.

3D stereoscopic image generation system 100 according to an embodiment of the present invention is the input image unit 200, the image acquisition unit 300, the viewer unit 400, the viewer posture collection unit 500, three-dimensional stereoscopic image The generation unit 600 and the stereoscopic display unit 700 may be included.

The input image unit 200 provides an input image from the outside. The input image unit 200 plays back image contents stored from an image media reproducing apparatus, provides an image from an image capturing apparatus, or provides an image from an image generating apparatus such as a computer, a game machine, or a complex multimedia device.

The image acquisition unit 300 processes an image provided from the input image unit 200 to obtain an image required for generating a 3D stereoscopic image.

The viewer unit 400 provides a viewer's attitude for watching a 3D stereoscopic image. The viewer's attitude can be provided from the stereoscopic glasses worn by the viewer. In addition, it is possible to provide a viewer's posture only by using features such as binocular, nose, and mouth of the viewer's face without using stereoscopic glasses.

The viewer posture collecting unit 500 collects the viewer posture provided by the viewer unit 400 and calculates the viewing direction and the viewing position of both eyes of the viewer.

The 3D stereoscopic image generating unit 600 converts the image obtained from the image capturing unit 300 into a 3D stereoscopic image by using the viewing direction and viewing position of the viewer provided by the viewing posture collecting unit 500.

The stereoscopic display unit 700 visualizes the stereoscopic image generated by the 3D stereoscopic image generating unit 600 so that the viewer can stereoscopically recognize the stereoscopic image.

2 illustrates an input image unit 200, an image acquisition unit 300, a viewer unit 400, a viewer posture collection unit 500, and a three-dimensional image forming a three-dimensional stereoscopic image generation system 100 reflecting a viewer's attitude. A block diagram showing a processing flow of the stereoscopic image generating unit 600 is shown.

The image used for input by the input image unit 200 may include a general 2D image 210, a general 3D image 220, a real time 3D rendering 230, and a 3D 3D image 240. The general two-dimensional image 210 includes a broadcast signal such as airwaves and cable TV, an output signal of a playback device of stored video content, and a video signal of a game machine, a computer, and a multimedia player, and the like. The general stereoscopic image 220 includes a stereoscopic image composed of a left eye image and a right eye image used in a stereoscopic image content such as a general 3D television and theater stereoscopic display device or a 3D Blu-ray disc. The real-time three-dimensional rendering 230 is an image generated by rendering an image in real time using computer graphics. The image is displayed in real time when the image is displayed in a three-dimensional image generating system instead of a previously produced and stored form. It includes an image generated through a computer graphics rendering pipeline included in the same device. Unlike the general stereoscopic image 220, the 3D stereoscopic image 240 includes a range image (range image or range video), and is an image including information about a distance from a camera photographing an image to a subject. The 3D stereoscopic image 240 may be composed of a combination of a general 2D image and a distance image. The 3D stereoscopic image 240 may also be composed of a combination of a general stereoscopic image and a distance image. Another example of the three-dimensional stereoscopic image 240 is a jet buffer that stores information about the distance between the general two-dimensional image generated as a result of real-time three-dimensional rendering and the pixel rendered from the camera used in the computer graphics rendering process ( Z buffer) may be composed of a combination of images. The image used for input in the input image unit 200 is not limited to the above-mentioned content, it is also possible to configure the combination of the above-mentioned image or the reverse of each component. For example, the input image may be composed of a combination of the general image supplied from the real-time three-dimensional rendering 230 and the distance image of the three-dimensional stereoscopic image 240.

The image preprocessor 310 performs preprocessing of an externally input image to facilitate generation of a 3D stereoscopic image according to the type of the image provided by the input imager 200. As an example of the role played by the image preprocessing unit 310, when the general 2D image 210 is provided from the input image unit 200, the main image 320 corresponding to the original image is additionally added through image processing. The auxiliary image 330 to be used is generated. The auxiliary image 330 may include brightness, sharpness, sharpness, focus, and movement displacement, color, and character of each pixel of the general 2D image 210. The distance image is generated by using the result of estimating the distance from the camera to each pixel of the original image using the characteristics of the image such as the recognition of the features of the background shape and the characteristics of the image using the characteristics of the caption or the text. It may be used as the auxiliary image 330.

As another example of the role played by the image preprocessor 310, when the general stereoscopic image 220 is provided by the input imager 200, the main image 320 and the secondary image corresponding to the original image may be processed through image processing. 330). The main image 320 may be selected from a left eye image and a right eye image of the input stereoscopic image 220 according to the characteristics of the dominance and recessiveness of both eyes of the viewer. For example, when both viewers have left eye dominance, the left eye image of the input general stereoscopic image 220 may be used as the main image 320. Another example of generating the main image 320 is to calculate the displacement difference of each pixel constituting the same subject from the left and right eye images of the input general stereoscopic image 220 to capture the left and right eye images or to render a computer graphic. By calculating the position of the camera at the time of generating the input image through the process, the camera position of the left eye image and the right eye image are calculated, and the image viewed by the virtual camera located at the center of the two cameras is generated. It can be used as an image 320. As an example of generating the auxiliary image 330 when the general stereoscopic image 220 is provided from the input image unit 200, each pixel constituting the same subject from the left eye image and the right eye image of the input general stereoscopic image 220. Calculate the camera position of the left eye image and the right eye image by calculating the position of the camera at the time of taking the left eye image and the right eye image using the displacement difference of The distance image of each pixel constituting the subject from the position of the camera of the image used in operation 320 may be calculated to generate a distance image, and may be used as the auxiliary image 330.

As another example of the role played by the image preprocessor 310, when the real-time 3D rendering 230 image is provided by the input image unit 200, the rendered image is processed in the same manner as the general 2D image 210. can do. Another example of the case where the real-time 3D rendering 230 image is provided by the input image unit 200 is to use the rendered image as the main image 320 and stored in a Z-buffer used in a computer graphic rendering process. The distance image may be used as the auxiliary image 330. As another example when the real-time 3D rendering 230 image is provided by the input image unit 200, the rendered image is used as the main image 320, and a distance image is separately generated during the computer graphic rendering process. 330).

As another example of the role played by the image preprocessor 310, when the 3D stereoscopic image 240 is provided by the input image unit 200, the general 2D image of the 3D stereoscopic image is converted into the main image 320. The distance image of the 3D stereoscopic image may be used as the auxiliary image 330.

3 is a diagram illustrating an example of an image acquisition process for authoring the 3D stereoscopic image 240 described above. The camera device 251 of FIG. 3 is a device for authoring a three-dimensional stereoscopic image 240 and a distance camera that records the distance of the subject 252 from the camera 251 at the same time as the camera photographing a general two-dimensional image. It is composed of a device capable of recording a distance such as a camera. Unlike the general stereoscopic image 220 photographing the left eye image and the right eye image using two general cameras, the apparatus for authoring the 3D stereoscopic image 240 may generate the general image and the distance image. As another example of an image acquisition process for authoring the 3D stereoscopic image 240, a general 2D image and a distance image may be generated through a computer graphic rendering process.

The viewer posture information 410 includes information for extracting a viewing direction and a viewing position of the viewer 400 viewing the 3D stereoscopic image. Examples of the viewer attitude information 410 include a feature of a viewer's face, a shape of general glasses worn by the viewer, a shape of stereoscopic glasses worn by the viewer for viewing a 3D stereoscopic image, and special glasses that the viewer can wear. The shape of the, includes a direction and position sensor included in the special glasses that the viewer can wear.

4 is a view showing an example of the three-dimensional glasses 420 for collecting the viewing attitude information 410 of one embodiment of the present invention. The viewer posture information 410 captures the shape of the front frame 421 of the general glasses or the stereoscopic glasses 420 from a camera included in the stereoscopic display unit 700 or a separate camera to perform image processing or a computer having a pattern recognition function. The image processing with the vision function can calculate the tilted direction and the position in the three-dimensional space of ordinary glasses or stereoscopic glasses. As another example of the viewer attitude information 410, an image processing or computer vision function having a pattern recognition function by photographing a face of a viewer watching a 3D stereoscopic image from a camera included in the stereoscopic display unit 700 or a separate camera is provided. Through image processing with a feature of the face of the viewer, such as eyes, nose, mouth can be recognized to estimate the inclination direction and position of the viewer's head in the three-dimensional space. As another example of the viewer attitude information 410, the tilted direction and position of the viewer's head using general glasses worn by the viewer watching the 3D stereoscopic image, the stereoscopic glasses 420, or a sensor mounted in a separate device Can be calculated. The above-mentioned general glasses, three-dimensional glasses 420 or a sensor mounted on a separate device is a gravity sensor that can measure the tilt of the device using gravity and a gyro sensor that can measure the tilt of the device using inertia. It may include all or only one of the two sensors. The type of sensor that may be included in the general glasses, the stereoscopic glasses 420 or a separate device in order to measure the information on the viewer's viewing attitude is not limited to the examples mentioned above. It can be made into a form that can be included or simply attached. The sensor mentioned above may be included in the three-dimensional glasses as shown in 422 of FIG. In order to facilitate the collection of the viewer's attitude information 410, a marker that can be recognized may be attached to the front surface of the general glasses or the stereoscopic glasses in the image processing process having a pattern recognition or computer vision function.

The viewer posture calculation unit 510 acquires the posture information of the viewer provided from the viewer posture information 410 and calculates the viewing direction position information for use in calculating the 3D stereoscopic image. Examples of functions performed by the viewer posture calculation unit 510 include binocular and nose of the viewer acquired in the viewer posture information 410 through image processing having a pattern recognition function or image processing having a computer vision function. The information about the viewing direction and the viewing position of the viewer can be calculated from the image including the feature of the face such as the mouth. Another example of a function performed by the viewer posture calculation unit 510 is a frame of general glasses or stereoscopic glasses in the viewer posture information 410 through image processing having a pattern recognition function or image processing having a computer vision function. Using the feature of the shape having the shape it is possible to calculate the information on the viewing direction and viewing position of the viewer from the image containing the normal glasses or stereoscopic glasses. As another example of the function performed by the viewer posture calculation unit 510, information about the viewing direction and the viewing position of the viewer may be calculated by obtaining the value (s) measured from the sensor (s) mentioned above. The function of the viewer posture calculation unit 510 is not limited to the above-mentioned contents, and only a part of the above-mentioned contents may be used or combined as necessary.

The 3D stereoscopic image calculating unit 610 uses the viewing direction position information 520 calculated by the viewing posture calculating unit 510 to provide the main image 320 and the auxiliary image 330 provided by the image obtaining unit 300. And generating a 3D stereoscopic image in response to the viewing direction and the viewing position of the viewer. In general, stereoscopic image using binocular disparity assumes the position of the left eye and the right eye horizontally for calculating the left eye image and the right eye image, and implements the stereoscopic image through horizontal photographing and horizontal reproduction. Unlike general stereoscopic images, the 3D stereoscopic image calculation unit 610 of the present invention calculates the inclination and displacement of the position of the left eye and the position of the right eye according to the inclination and the position of the head of the viewer when generating the left eye image and the right eye image. The direction and magnitude of the displacement of the left eye image and the right eye image can be calculated. A method of calculating a three-dimensional stereoscopic image composed of a left eye image and a right eye image from the main image 320 and the auxiliary image 330 is described as a specific embodiment for carrying out the invention in Korean Patent Nos. 10-0381817 and 10-0538471. The method described above or the embodiment described in the patent (s) may be modified.

5 and 6, the left eye image 620 and the right eye image 630 constituting the 3D stereoscopic image are generated by reflecting the viewing direction position information 520 from the main image 320 and the sub image 330. How to do this will be described in detail.

FIG. 5 illustrates a situation in which the viewer views the subject horizontally from FIG. 3 illustrating an authoring concept of the three-dimensional stereoscopic image 240, which is an example of the main image 320 and the auxiliary image 330 used in the image acquisition unit 300. It is a drawing expressed on the assumption. The camera plays the role of the viewer's eyes in the filming process using the camera or the video authoring process using the computer graphic rendering. Given the camera model commonly used in computer graphics, it's easy to understand. When a general two-dimensional image generated from the reference position 254 of the camera used to generate the image is shown to the viewer as it is, the effect of looking at the subject 252 at the same position 254 with one eye can be obtained. A simple method of generating a three-dimensional stereoscopic image from a general two-dimensional image, as shown in FIG. 5, sets the reference position 431 of the left eye (or right eye) of the viewer and the reference position 254 of the camera of the main image 320. Make it the same and place the viewer's right eye (or left eye) horizontally with the left eye (or right eye). The above-mentioned case is similar to the stereoscopic image generation method using the jet buffer described in FIGS. 2A, 2B, 2C, and the embodiment of Patent No. 10-0381817. More specifically, assume that the reference position 431 of the viewer's left eye, which is the same as the reference position 254 of the camera, is E (L) representing the left eye of FIG. 2A of Patent 10-0381817, and reference position 432 of the viewer's right eye is defined. It may be assumed that E (R) represents the right eye of FIG. 2A of Patent 10-0381817. In addition, the main image 320 mentioned above is assumed to be a two-dimensional plane image by monocular described in Patent 10-0381817, and the distance image stored in the auxiliary image 330 is described in Patent 10-0381817. It can be assumed as distance information of an image stored in a buffer. In addition, it may be assumed that one vertex 253 of the subject 252 of FIG. 5 is point P of FIG. 2A of Patent No. 10-0381817. The assumption mentioned above is clear to those skilled in the art because the position of the left and right eyes of FIG. 5 and the reference position of the camera of the main image are based on the similarities to those of FIGS. 2A, 2B and 2C of Patent 10-0381817. You will agree. Therefore, when the left and right eyes of the viewer shown in FIG. 5 are horizontal and one eye of the viewer is the same as the reference position of the main image 320, the 3D stereoscopic image calculation unit 610 generates a 3D stereoscopic image. It will be apparent to those skilled in the art that the stereoscopic image generating method described in Patent 10-0381817 and Patent 10-0381817, which is cited in Patent 10-0381817, can be performed.

FIG. 6 illustrates a concept of authoring a three-dimensional stereoscopic image 240, which is an example of a main image 320 and an auxiliary image 330 used in the image acquisition unit 300, and a viewer selects a subject at an arbitrary position. It is assumed that the situation is viewed in the inclined direction of the drawing. Considering the viewer's head viewing in an arbitrary posture, the viewer's left eye reference position 441 and the right eye reference position 442 are different from the camera's reference position 254 of the main image 320 in an arbitrary direction. Can be tilted As described above, when the reference position 441 of the viewer's left eye and the reference position 442 of the right eye are different from the reference position 254 of the camera of the main image 320, the auxiliary image 330 is obtained from the main image 320. The 3D stereoscopic image calculation unit 610 may calculate and generate the left eye image 620 and the right eye image 630, respectively. For the clear understanding of those skilled in the art, the process of generating the left eye image 620 will first be described. One vertex 253 of the subject 252 of FIG. 6 may be assumed to be the point P of FIG. 2A of Patent 10-0381817. The reference position 254 of the camera of FIG. 6 may be assumed to be the position E (L) of the left eye of FIG. 2A of patent 10-0381817. It can be assumed that the reference position 441 of the left eye of the viewer of FIG. 6 is the position E (R) of the right eye of FIG. 2A of Patent No. 0381817. From the above assumption, Patent No. 10-0381817 uses similarities between triangles connecting 253, 254, and 441 of FIG. 6 and triangles connecting P, E (L), and E (R) of FIG. 2A of Patent 10-0381817. And a three-dimensional image generation method of Patent 10-0538471 citing Patent 10-0381817 may be performed by the three-dimensional stereoscopic image calculation unit 610. The main image 320 and the auxiliary image 330 generated by the image acquisition unit 300 are used as distance information of the two-dimensional plane image and the image stored in the Z buffer by the monocular described in the stereoscopic image generation method of Patent 10-0381817. The 3D stereoscopic image calculator 610 may calculate the left eye image 620. Similarly to the above description, the process of generating the right eye image 630 will now be described for a clear understanding by those skilled in the art. One vertex 253 of the subject 252 of FIG. 6 may be assumed to be the point P of FIG. 2A of Patent 10-0381817. The reference position 254 of the camera of FIG. 6 may be assumed to be the position E (L) of the left eye of FIG. 2A of patent 10-0381817. The reference position 442 of the viewer's right eye in FIG. 6 may be assumed to be the position E (R) of the right eye in FIG. 2A of patent 0381817. From the above assumption, Patent No. 10-0381817 utilizes the similarity between the triangle connecting 253, 254 and 442 of FIG. 6 and the triangle connecting P, E (L) and E (R) of FIG. 2A of Patent 10-0381817. And a three-dimensional image generation method of Patent 10-0538471 citing Patent 10-0381817 may be performed by the three-dimensional stereoscopic image calculation unit 610. The main image 320 and the auxiliary image 330 generated by the image acquisition unit 300 are used as distance information of the two-dimensional plane image and the image stored in the Z buffer by the monocular described in the stereoscopic image generation method of Patent 10-0381817. The right eye image 630 may be calculated by the 3D stereoscopic image calculator 610. The above description will be clearly understood by those skilled in the art including the concept of the drawings shown in FIG. 6 and the similarity between the embodiments of the stereoscopic image generating method of Patents 10-0381817 and 10-0538471.

The performance of the 3D stereoscopic image calculation unit 610 is not limited to the above-mentioned functions, and other functions not mentioned above may be clearly understood by those skilled in the art through the embodiments of Patent 10-0381817 and Patent 0538371. Since there will be, it will be omitted in the description of this embodiment.

7 to 9 are left eye images generated by the 3D stereoscopic image calculation unit 610 by reflecting the viewing direction position information 520 calculated from the viewing position information 410 on the image acquired by the image obtaining unit 300 ( 620 and the right eye image 630 are visualized on the stereoscopic display unit 700 to show the principle shown to the viewer.

An example of the stereoscopic display unit 700 may include a hardware device 710 and a stereoscopic glasses 420 for visualizing a stereoscopic image. Another example of the stereoscopic display unit 700 may be a hardware device that visualizes a stereoscopic image that does not require glasses. Examples of the stereoscopic display unit 700 are not limited to the above description.

FIG. 7 illustrates that the viewer 400 according to an embodiment of the present invention wears the stereoscopic glasses 420 and reflects the viewer's attitude viewed horizontally from the front of the stereoscopic display unit 700. FIG. 3 shows an example of a stereoscopic display unit 700 in which a left eye image and a right eye image are visualized. One edge 720 of the left eye image 620 shown in the hardware device 710 for visualizing a stereoscopic image includes a left vertex 721 and a right vertex 722. One corner 730 of the right eye image 630 includes a left vertex 731 and a right vertex 732. The left eye image 620 and the right eye image 630 are projected to the viewer's left and right eyes, respectively, to a corner 820 of the virtual object 810 in the space in front of the viewer's eyes through a stereoscopic recognition process by binocular parallax. It can be perceived by the viewer. One edge 820 is composed of a left vertex 821 and a right vertex 822. As an example of the visualization process of the 3D stereoscopic image shown in FIG. 7, the position of the left and right eyes of the viewer is located horizontally, which is similar to the general stereoscopic visualization process.

8 is a view of the left eye image generated by the 3D stereoscopic image generator by reflecting a viewer's attitude in which the viewer 400 tilts his head 45 degrees to the right from the front of the stereoscopic display unit 700 according to an embodiment of the present invention; FIG. 3 is a diagram illustrating an example of a stereoscopic display unit 700 in which a right eye image is visualized. One corner 740 of the left eye image 620 shown in the hardware device 710 for visualizing the stereoscopic image is composed of a left vertex 741 and a right vertex 742. One corner 750 of the right eye image 630 includes a left vertex 751 and a right vertex 752. The left eye image 620 and the right eye image 630 are projected to the viewer's left and right eyes, respectively, to a corner 840 of the virtual object 830 in the space in front of the viewer's eyes through a stereoscopic recognition process by binocular parallax. It can be perceived by the viewer. One edge 840 consists of a left vertex 841 and a right vertex 842. An example of the visualization process of the 3D stereoscopic image shown in FIG. 8 is that the head of the viewer is inclined 45 degrees to the right according to the viewer's posture, unlike the general stereoscopic visualization process, and thus the left eye image 620 and the right eye image ( One edge 840 of the object 830 visualized by 630 is present at the position 740 in the left eye image and at the position 750 in the right eye image. Comparing the difference in the direction and the size of the displacement of one corner 740 and the other one edge 750 of FIG. 8 with the difference in the direction and the size of the displacement of one corner 720 and the other one edge 730 of FIG. As follows. Although the displacement of FIG. 7 exists only in the horizontal direction, it can be seen that the displacement shown in FIG. 8 is inclined in the 45 degree direction similar to the viewing direction. From the above description, those skilled in the art will clearly understand that the method of generating a 3D stereoscopic image of the present invention differs from a general stereoscopic image generating method in which a difference between a left eye image and a right eye image is displayed only in the horizontal direction.

FIG. 9 illustrates a left eye generated by the 3D stereoscopic image generator by reflecting a viewer's attitude viewed vertically by turning the head 90 degrees to the right from the front of the stereoscopic display unit 700 according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a stereoscopic display unit 700 in which an image and a right eye image are visualized. One corner 760 of the left eye image 620 shown in the hardware device 710 for visualizing a stereoscopic image is composed of a left vertex 761 and a right vertex 762. One corner 770 of the right eye image 630 includes a left vertex 771 and a right vertex 772. The left eye image 620 and the right eye image 630 are projected to the left and right eyes of the viewer, respectively, to a corner 860 of the virtual object 850 in the space in front of the viewer's eyes through a stereoscopic recognition process by binocular parallax. It can be perceived by the viewer. One edge 860 consists of a left vertex 861 and a right vertex 862. An example of the visualization process of the 3D stereoscopic image illustrated in FIG. 9 is an object visualized by the left eye image 620 and the right eye image 630 because the viewer's head is inclined 90 degrees to the right according to the viewer's viewing attitude. One corner 860 of 850 is located at position 760 in the left eye image and position 770 in the right eye image. When the left and right eyes are positioned in the vertical direction according to the viewer's viewing attitude, it is clear that one corner 760 of the left eye image 620 and one corner 770 of the right eye image 630 have displacements in the vertical direction. Will understand. From the above description, those skilled in the art will appreciate that the three-dimensional stereoscopic image generation method of the present invention reflects the positions of structural units such as points, lines, and surfaces constituting the left and right eye images according to the viewing direction and viewing position of the viewer. It will be clearly understood.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, Of course.

When the general stereoscopic image 220 is provided by the input image unit 200 and the viewing direction calculated by the viewer posture collector 500 is horizontal, the left eye image provided by the 3D stereoscopic image generator 600 from the original image is provided. And right eye images can be used as a stereoscopic image.

Another example of the case where the real-time 3D rendering 230 image is provided by the input image unit 200 is to reflect the viewing direction position information 520 in real time to display the left eye image 620 and the right eye image 630 in real time. Can be generated directly in the rendering 230.

100: 3D stereoscopic image generation system
200: input image unit, 210: normal 2D image, 220: normal stereoscopic image, 230: real time 3D rendering, 240: 3D stereoscopic image, 251: camera, 252: subject, 253: one vertex of the subject, 254: Reference position of camera in three-dimensional space
300: image acquisition unit 310: image preprocessing unit 320: main image 330: auxiliary image
400: viewer part, 410: viewing attitude information, 420: stereoscopic glasses, 421: stereoscopic glasses frame, 422: posture measuring device, 430: glasses for showing the position of both eyes of the viewer, 431: reference position of the viewer's left eye, 432: reference position of the viewer's right eye, 440: glasses to show the position of both viewers' eyes, 441: reference position of the viewer's left eye, 442: reference position of the viewer's right eye
500: viewing posture collecting unit, 510: viewing posture calculating unit, 520: viewing direction location information
600: 3D stereoscopic image generation unit, 610: 3D stereoscopic image calculation unit, 620: left eye image, 630: right eye image
700: stereoscopic display unit, 710: stereoscopic display, 720: an example of a line visualized as part of a left eye image, 721: a left vertex of 720, 722: a right vertex of 720, 730: an example of a line visualized as part of a right eye image, 731: 730 Left vertex of, 732: right vertex of 730, 740: example of line visible as part of left eye image, 741: left corner of 740, 742: right corner of 740, 750: example of line visible as part of right eye image, 751: 750 Left vertex of, 752: right vertex of 750, 760: example of a line visualized as part of the left eye image, 761: left corner of 760, 762: right corner of 760, 770: example of line visible as part of the right eye image, 771: 770 Left corner of, 772: Right corner of 770
810: virtual subject visualized in three-dimensional stereoscopic image, 820: one corner of the virtual subject, 821: left vertex of one corner of the virtual subject, 822: right vertex of one corner of the virtual subject, 830: three-dimensional Virtual subject visualized as a stereoscopic image, 840: one corner of the virtual subject, 841: left vertex of one corner of the virtual subject, 842: right vertex of one corner of the virtual subject, 850: visualized with three-dimensional stereoscopic image Virtual subject, 860: one corner of the virtual subject, 861: left vertex of one corner of the virtual subject, 862: right vertex of one corner of the virtual subject

Claims (8)

An image acquisition unit which processes an image input from the outside and obtains a main image and an auxiliary image required to generate a 3D stereoscopic image;
A viewer posture collecting unit configured to collect viewer postures from viewers viewing 3D stereoscopic images to calculate viewer posture information of both eyes of the viewer; And
It includes a three-dimensional stereoscopic image generating unit for generating a three-dimensional stereoscopic image by reflecting the viewing attitude information of both eyes of the viewer calculated by the viewing posture collector from the main image and the auxiliary image processed from the image acquisition unit,
The stereoscopic image generated by the 3D stereoscopic image generating unit is a 3D stereoscopic image generating system in which a left eye image and a right eye image change according to a viewer's viewing attitude. The method of claim 1, wherein the image acquisition unit,
A main image used as a main image for generating a 3D stereoscopic image from an image input from the outside;
3D stereoscopic image generation system for generating an auxiliary image representing distance information of an object in the main image The method of claim 1, wherein the viewer posture collecting unit,
A three-dimensional stereoscopic image generation system extracting feature information on the viewer's face, general glasses, or stereoscopic glasses from the image of the viewer's face, and calculating the viewer's attitude using the extracted feature information. The method of claim 1, wherein the viewer posture collecting unit,
A three-dimensional stereoscopic image generation system for calculating a viewer's viewing attitude from information received from a gyro sensor or a gravity sensor attached to general glasses or stereoscopic glasses worn by the viewer, or a gyro sensor or gravity sensor made in the form of an attachment The apparatus of claim 1, wherein the external image input to the image acquisition unit comprises: a main image formed of a general two-dimensional image;
3D stereoscopic image generation system including an auxiliary image consisting of a distance image stored distance information for each pixel of the main image delete delete An image acquisition method of acquiring a main image and an auxiliary image required for generating a 3D stereoscopic image by processing an image input from the outside;
A viewer posture collection method of collecting viewer postures from viewers viewing 3D stereoscopic images to calculate viewer posture information of both eyes of the viewer; And
And a method of generating a 3D stereoscopic image by reflecting viewer attitude information of both viewers calculated by the viewer attitude collection method from the main image and the auxiliary image processed by the image acquisition method.
The 3D stereoscopic image generated by the 3D stereoscopic image generating method reflects the viewer's attitude in which the left eye image and the right eye image change according to the viewer's viewing attitude.

Images