KR20090037247A - Method and device for transformation from multi focused 2d image to 3d image, and recording media - Google Patents

Abstract

A method and a device for converting 3d video by using multi focus 2d video and a recording medium therefor are provided to have a function of converting plural photographed multi focus 2D videos into 3D video, thereby producing cheap and natural 3D video. A video photographing unit(150) photographs a subject through one lens. A focus information determination unit(160) reads preview video data. A focus control unit(155) automatically focuses a focus location determined according to the number of focuses. The video photographing unit photographs multi focus 2D video. The focus information determination unit reads preview video data. The focus information determination unit N focus location information with regard to the subject. A video converting module(100) generates right and left eye video data. A screen output unit(180) scales down and outputs the video data at a scaled down ratio.

Description

Method and device for transforming 3D stereoscopic image using multi-focus 2D image and recording medium {Method and Device for Transformation from Multi Focused 2D Image to 3D Image, and Recording Media}

1 is a diagram illustrating a configuration of a 3D stereoscopic image conversion apparatus using a multifocal 2D image according to an exemplary embodiment of the present invention.

2A and 2B illustrate a method of determining focus information for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

3A and 3B illustrate a focal length calculation method for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

4A and 4B illustrate a focus overlapping process for converting 3D stereoscopic images using a multifocal 2D image according to an exemplary embodiment of the present invention.

5A and 5B illustrate a depth value calculation process for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

FIG. 6 is a diagram illustrating a distance recognition method for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

7 is a diagram illustrating a parallax processing method for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

8A and 8B illustrate a 3D stereoscopic image conversion process using a multifocal 2D image according to an exemplary embodiment of the present invention.

9A and 9B illustrate a 3D stereoscopic image conversion process using a multifocal 2D image according to another exemplary embodiment of the present invention.

<Description of main parts of drawing>

100: image conversion module 105: input image reduction unit

110: focal length calculation unit 115: focus overlap processing unit

120: depth value calculator 125: depth value image synthesis unit

130: reduction ratio calculation unit 135: parallax processing unit

140: left and right image generating unit 145: 2D image recording module

150: image capture unit 155: focus control unit

160: focus information determiner 165: image acquisition unit

170: raw image storage unit 175: key input unit

180: screen output unit 183: storage medium

185: 3D image output module 190: image output unit

193: Image storage unit 195: Communication unit

The present invention provides a method of determining N (N> 1) focal positions and focal number for a two-dimensional image, and N (N> 1) corresponding to the determined N (N> 1) focal positions and focal number. Computing N (N> 1) focal lengths for the N (N> 1) multifocal two-dimensional image data when multi-focus two-dimensional image data is obtained, and calculating the calculated N (N> 1). Calculating N (N> 1) depth values in inverse proportion to the focal lengths, synthesizing N (N> 1) multi-focus two-dimensional image data according to N (N> 1) depth values, and In addition, among the N (N> 1) multi-focus two-dimensional image data, zero parallax image data, positive parallax image data, and negative parallax image data are identified, and the identified Checks the bi-parallel direction pixel movement information based on the parallax angles for at least one bi-division image data based on the zero parallax Checking the disparity direction pixel shift information based on the disparity angle of the at least one disparity image data based on the zero parallax, and applying the identified disparity direction pixel shift information to the at least one disparity image data And generating left eye image data and right eye image data by applying the identified parallax direction pixel shift information to at least one or more parallax image data. It is about.

The method of converting a 2D image into a 3D stereoscopic image includes converting a conventional 2D image into a 3D stereoscopic image and converting a 2D image photographed for a 3D stereoscopic image into a 3D stereoscopic image. Can be distinguished.

Here, the conventional method of converting a two-dimensional image to a three-dimensional stereoscopic image, correcting the sharpness of the two-dimensional image (or block unit image) taken by a single lens through image processing, and the parallax processing of the two-dimensional image For this purpose, a method of moving a pixel by a predetermined size from side to side and converting the pixel into a 3D stereoscopic image is generally used.

However, in the method of converting the 2D image into the 3D stereoscopic image, the 3D image is shifted by a predetermined size to the left and right of the entire 2D image (or block unit image) without considering the focal length (or depth value). Because it converts to a stereoscopic image, it includes a problem that causes severe dizziness for viewers who are viewing 3D stereoscopic images.

In addition, a method of converting a 2D image photographed for a 3D stereoscopic image into a 3D stereoscopic image may include two images of a single subject through two cameras or two lenses. A method of photographing and converting the two captured images into a three-dimensional stereoscopic image is generally used.

However, since the method of converting the 2D image photographed for the 3D stereoscopic image into the 3D stereoscopic image requires an expensive camera (or lens), a general digital camera (or video camera) is provided. Contains esoteric issues.

An object of the present invention for solving the above problems is a focal number determiner for determining N (N> 1) focal positions and focal number for a two-dimensional image, and N for N (N> 1) focal positions. A focus control unit for controlling the multi-focus by (N> 1) focal number, an image acquisition unit for acquiring N (N> 1) multi-focus two-dimensional image data by the focus control, and N (N> 1) A focal length calculator for calculating N (N> 1) focal lengths for the multi-focal two-dimensional image data, and N (N> 1) depth values in inverse proportion to the calculated N (N> 1) focal lengths A depth value calculating unit for calculating a P, a depth value image combining unit for synthesizing N (N> 1) multi-focus two-dimensional image data according to N (N> 1) depth values, and N (N> 1) multiples Of the two-dimensional image data, zero parallax image data, positive parallax image data, and negative parallax image data are identified. Checks pixel direction information in both directions based on the disparity angles of the at least one billax image data based on the identified zero parallax, and disparity the at least one disparity image data based on the identified zero parallax A parallax information confirming unit that verifies the parallax direction pixel movement information based on an angle, and applies the identified bilateral parallax direction pixel movement information to at least one or more parallax image data, and applies the identified parallax direction pixel movement information to at least The present invention provides a three-dimensional stereoscopic image conversion apparatus using a multi-focus two-dimensional image having a left and right image generating unit for generating left eye image data and right eye image data by applying to one or more disparity image data.

According to an embodiment of the present invention, a method of converting a 3D stereoscopic image using a multifocal 2D image includes determining N (N> 1) focal positions and a number of focal points for a 2D image, and determining the determined N (N> 1). When N (N> 1) multi-focus two-dimensional image data corresponding to three focal positions and the number of foci are acquired, N (N> 1) focal points for the N (N> 1) multi-focus two-dimensional image data Calculating a distance; calculating N (N> 1) depth values in inverse proportion to the calculated N (N> 1) focal lengths; and N (N> 1) depth values according to N (N> 1) depth values. N> 1) synthesizing the multi-focus two-dimensional image data, and among the N (N> 1) multi-focus two-dimensional image data, zero parallax image data and positive parallax image data And negative parallax image data, and at least one positive parallax image data based on the identified zero parallax. Confirming the bilateral parallax direction pixel movement information on the basis of the difference, and identifying the parallax direction pixel movement information based on the parallax angle for at least one parallax image data based on the identified zero parallax; Generating left eye image data and right eye image data by applying the bilateral parallax direction pixel shift information to at least one positive parallax image data, and applying the identified parallax direction pixel shift information to the at least one parallax image data; It is characterized by comprising.

In the 3D stereoscopic image conversion method using the multifocal 2D image according to the present invention, the N (N> 1) multifocal 2D image data corresponding to the determined N (N> 1) focal positions and the number of focal points The method may further include converting the size (or resolution) of the N (N> 1) two-dimensional image data into a preset conversion size (or resolution).

The three-dimensional stereoscopic image conversion method using the multi-focus two-dimensional image according to the present invention, further comprising the step of overlapping so that the focal position of the N (N> 1) two-dimensional image data aligned in accordance with the line of sight direction It is done.

In the 3D stereoscopic image conversion method using the multifocal 2D image according to the present invention, the horizontal distortion of the left and right image data is corrected based on the disparity angle information and the focal length information corresponding to the left eye image data and the right eye image data. Comprising the step of calculating the reduction ratio to be corrected.

On the other hand, it includes a recording medium recording a program for executing the three-dimensional stereoscopic image conversion method using the above-described multi-focus two-dimensional image.

On the other hand, the three-dimensional stereoscopic image conversion apparatus using a multi-focus two-dimensional image according to the present invention, the focus number determination unit for determining the N (N> 1) focus positions and the number of focus for the two-dimensional image, N (N A focus control unit for controlling multiple focuses by N (N> 1) focal points for> 1) focal positions, and image acquisition for acquiring N (N> 1) multi-focus two-dimensional image data by the focus control And a focal length calculator for calculating N (N> 1) focal lengths for N (N> 1) multi-focal two-dimensional image data, and inversely proportional to the calculated focal lengths of N (N> 1). A depth value calculating unit for calculating N (N> 1) depth values, a depth value image combining unit for synthesizing N (N> 1) multi-focus two-dimensional image data according to N (N> 1) depth values, and Zero parallax image data, positive parallax image data, and negative parallax among N (N> 1) multi-focus 2D image data Parallax) checks the image data, and checks the bi-parallel direction pixel movement information based on the disparity angles for at least one bi-division image data based on the identified zero parallax, and at least one on the basis of the identified zero parallax The parallax information confirming unit which checks the parallax direction pixel movement information based on the parallax angle with respect to the parallax image data above, and applies the identified bilateral parallax direction pixel movement information to at least one or more parallax image data. And a left and right image generating unit generating left eye image data and right eye image data by applying the disparity direction pixel shift information to at least one disparity image data.

A three-dimensional stereoscopic image conversion apparatus using a multi-focus two-dimensional image according to the present invention, the input for converting the size (or resolution) of the N (N> 1) two-dimensional image data to a preset conversion size (or resolution) Characterized in that it further comprises an image reduction unit.

The three-dimensional stereoscopic image converting apparatus using the multi-focus two-dimensional image according to the present invention further comprises a focus overlapping processing unit which overlaps so that the focal positions of N (N> 1) two-dimensional image data are aligned along the line of sight. It is characterized by.

According to an embodiment of the present invention, a 3D stereoscopic image conversion apparatus using a multifocal 2D image includes horizontal distortion of the left and right image data based on disparity angle information and focal length information corresponding to the left eye image data and the right eye image data. A reduction ratio calculation unit for calculating a reduction ratio to be corrected is further provided.

In the three-dimensional stereoscopic image conversion apparatus using a multi-focus two-dimensional image according to the present invention, the focus number determiner, pattern recognition (or image recognition) for N (N> 1) objects included in the two-dimensional image data Check whether the 2D image data includes N (N> 1) objects-a subject including a boundary distinct from a background (or other object) such as a person, an animal, or an object. If (N> 1) objects are identified, the center position of the identified object is determined as N (N> 1) focal positions. If the object is not confirmed as the result of the checking, the 2D image data is determined in a vertical direction ( Or in the left-right direction or the lattice direction), and divide the focus points into N (N> 1) focus regions, and determine the center positions of the divided focus regions as N (N> 1) focus positions.

Hereinafter, with reference to the accompanying drawings and description will be described in detail the operating principle of the preferred embodiment of the present invention. However, the drawings and the following description shown below are for the preferred method among various methods for effectively explaining the features of the present invention, the present invention is not limited only to the drawings and description below. In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. Terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users or operators. Therefore, the definition should be made based on the contents throughout the present invention.

In addition, preferred embodiments of the present invention to be carried out below are provided in each system functional configuration to efficiently describe the technical components constituting the present invention, or system functions that are commonly provided in the technical field to which the present invention belongs. The configuration will be omitted, and described mainly on the functional configuration to be additionally provided for the present invention. If those skilled in the art to which the present invention pertains, it will be able to easily understand the function of the components that are conventionally used among the omitted functional configuration not shown below, and also the configuration omitted as described above The relationship between the elements and the components added for the present invention will also be clearly understood.

In addition, the following examples will be used to appropriately modify, integrate, or separate the terminology so that those skilled in the art to which the present invention pertains may clearly understand the present invention. The present invention is by no means limited thereto.

As a result, the technical spirit of the present invention is determined by the claims, and the following examples are one means for efficiently explaining the technical spirit of the present invention to those skilled in the art to which the present invention pertains. It is only.

1 is a diagram showing the configuration of a three-dimensional stereoscopic image conversion apparatus using a multi-focus two-dimensional image according to an embodiment of the present invention.

In more detail, in FIG. 1, in a digital camera (or video camera) having an autofocus control function, a plurality of two-dimensional images having different focal lengths are photographed with respect to the same subject, and the plurality of photographed two-dimensional images are taken. A device configuration for generating left and right images corresponding to left and right eyes and converting them into 3D stereoscopic images based on a focal length and depth value for the image, and having a general knowledge in the art. If it is possible to infer various implementation methods for constructing a 3D stereoscopic image conversion apparatus using the multi-focus 2D image by referring to and / or modifying the drawing 1, the present invention includes all the inferred implementation methods. It is made by, and the technical features are not limited only to the implementation method shown in FIG.

For example, an embodiment of a 3D stereoscopic image converting apparatus using a multifocal 2D image shown in FIG. 1 captures a 2D image with a single lens and has a digital camera having an autofocus control function for the 2D image. (Or a video camera), it will be described as being provided, but the three-dimensional stereoscopic image conversion apparatus using the multi-focus two-dimensional image is not limited to the embodiment shown in FIG. The 3D stereoscopic image converting apparatus using the focus 2D image may be implemented in the form of an external device connected to the digital camera (or video camera), or in the form of a device (or server) connected to the digital camera (or video camera) through a communication network. It is to be understood that the present invention is not limited to this, it is possible to implement both.

In general, a digital camera (or video camera) for photographing a two-dimensional image with one lens controls an operation of the digital camera (or a video camera), or a two-dimensional image is captured through the digital camera (or a video camera). A key input unit 175 for generating key data for photographing, an image capturing unit 150 for photographing a 2D image of a subject through a single lens corresponding to the key data for capturing the 2D image, Image acquisition unit 165 for acquiring 2D image data corresponding to a 2D image captured by the image capturing unit 150 through a charge coupled device (CCD), and various types of digital cameras (or video cameras). A screen output unit 180 for outputting 2D image data acquired through a control screen or the image acquisition unit 165, and a low storing image 2D image data. It characterized in that obtained by having a medium (183), and further characterized in that comprising the further focus control unit 155 for controlling the auto focus function of the 2-D image.

Those skilled in the art will be familiar with the technical features of the digital camera (or video camera) configuration to shoot a two-dimensional image through the autofocus function, detailed description thereof Is omitted for convenience.

According to the present invention, the image capturing unit 150 has a high speed photographing function for converting a 3D stereoscopic image using the multi-focus 2D image, and the CCD sensitivity of the image capturing unit 165 is provided. It is characterized in that the high to facilitate the high-speed shooting, the focus control unit 155 is characterized in that for controlling the focus of the image capture unit 150 at a high speed to facilitate the high-speed shooting.

Referring to FIG. 1, the 3D stereoscopic image converting apparatus using the multi-focus 2D image is captured by the image capturing unit 150, and the preview image data obtained by the image capturing unit 165. A focus information determination unit 160 for determining at least one or more focus position information and focus number information of the subject photographed by the image capturing unit 150 and reading the determined focus position according to the determined focus number; A focus controller 155 which controls to focus automatically, and an image photographing unit which photographs a plurality of multi-focus two-dimensional images by automatically focusing the focus position of the subject by the number of the focal points under the control of the focus controller 155 ( 150 and an image acquisition unit 165 for acquiring a plurality of multi-focus two-dimensional image data corresponding to the multi-focus two-dimensional image photographed by the image capturing unit 150. 2D image capturing module 145 further comprising a raw image storage unit 170 for storing the plurality of multi-focus two-dimensional image data photographed in the storage medium 183 according to the intention of the skilled person. It is characterized by comprising a).

A digital camera (or video camera) equipped with a 3D stereoscopic image conversion function using the multifocal 2D image captures a subject with one lens through the image capturing unit 150, and the image capturing unit 165. ) To obtain preview image data of the photographed 2D image, wherein the focus information determiner 160 reads the preview image data to obtain an image of the subject photographed through the image photographing unit 150. N (N> 1) pieces of focal position information are determined, and the number of focal positions is determined as focal number information for the subject.

The focus information determiner 160 includes N (N> 1) objects (eg, a subject including a boundary distinct from a background (or another object) such as a person, an animal, or an object) in the preview image data. It is characterized by checking whether there is.

According to an exemplary embodiment of the present invention, the focus information determiner 160 automatically recognizes N (N> 1) objects included in the preview image data through pattern recognition (or image recognition). .

According to another exemplary embodiment of the present invention, the focus information determiner 160 manually selects N (N> 1) objects included in the preview image data in association with the key input unit 175. It is possible, and the invention is not limited thereby.

If N (N> 1) objects are included in the preview image data, the focus information determiner 160 sets N center positions of N (N> 1) objects included in the preview image data. And (N> 1) focal positions, wherein the number of N (N> 1) focal positions is determined as N (N> 1) focal positions.

According to the exemplary embodiment of the present invention, the N (N> 1) pieces of focal number information may include the maximum number of shots that the image capturing unit 150 can take for a unit time (eg, 50 ms, etc.) or the focus control unit 155. ) Can control the focus of the image capturing unit 150 for a unit time, or the image obtaining unit 165 can acquire 2D image data from the image capturing unit 150 for a unit time. It is preferable to set smaller than the limit focal number including at least one of the maximum number of image data acquisition possible.

According to an exemplary embodiment of the present disclosure, when the number of focus positions (or the number of focal points) included in the preview image data is greater than the limit focal number, the focus information photographing unit may include the preview image among the objects included in the preview image data. It is preferable to determine N (N> 1) focal positions for objects within the above limit focal number among objects at the center position (or top, bottom, left, or right) of the viewing image data.

According to another exemplary embodiment of the present invention, when the number of focus positions (or the number of focus points) included in the preview image data is less than the limit focus number, the focus information photographing unit includes N (N) included in the preview image data. It is desirable to determine N (N> 1) focal positions for> 1) objects.

On the other hand, when the identifiable object is not included in the preview image data (for example, a sky, a horizon, a plain without a mountainous terrain, etc.), the focus information determiner 160 stores the preview image data by N (N> 1). N) (N> 1) focus area, and the center of the N (N> 1) focus area is determined as N (N> 1) focus position, the number of N (N> 1) focus position Is determined as N (N> 1) focal number.

According to the exemplary embodiment of the present invention, the N (N> 1) pieces of focal number information may include the maximum number of shots that the image capturing unit 150 can capture for a unit time, or the focus control unit 155 captures the image for a unit time. Maximum number of control times for controlling the focus of the photographing unit 150 or maximum number of image data acquisition units for obtaining the 2D image data from the image photographing unit 150 by the image obtaining unit 165 for a unit time. It is preferably set smaller than the limit focal number including at least one.

According to the exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the vertical direction, and N ( It is preferable to determine the center position of N> 1) focal regions as N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the left and right directions, and the N divided in the left and right directions as described above. It is preferable to determine the center positions of the (N> 1) focal regions as N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions according to a grid shape, and divides the preview image data into a grid shape as described above. It is preferable to determine the center position of the N (N> 1) focusing areas as N (N> 1) focusing positions.

When the N (N> 1) focal position information and the N (N> 1) focal number are determined by the focus information determiner 160, the focus controller 155 may focus on the image capturing unit 150. It characterized in that the control to auto-focus to the first focusing position of the N (N> 1) of the focusing position, the image capture unit 150 is to take a two-dimensional image of the first focusing position The image acquisition unit 165 may obtain first two-dimensional image data corresponding to the first two-dimensional image.

Thereafter, the focus controller 155 automatically focuses the image capturing unit 150 to the nth (n = 2, 3, ..., N) focusing positions among the N (N> 1) focusing positions. It characterized in that the control to focus, the image capturing unit 150 is characterized in that for taking a two-dimensional image of the n (n = 2, 3, ..., N) focus position, the image The obtaining unit 165 acquires n (n = 2,3, ..., N) 2D image data corresponding to the n (n = 2,3, ..., N) 2D image. It is characterized by.

According to the present invention, the focus control unit 155 controls the first to nth (n = 2,3, ..., N) focus positions to be autofocused, while the first to nth (n = 2, 3, ..., N) focus control information for the focus position is generated, wherein the focus control information is the first to n th (n = 2, 3, ..., N) focus And lens separation distance information for autofocusing a position, and refractive index information (or magnification information) of the spaced lens.

According to an exemplary embodiment of the present invention, when the digital camera is provided with a 3D stereoscopic image converting apparatus using the multifocal 2D image, N (N> 1) focal positions determined by the focus information determiner 160 The focus control unit 155 controls the auto focus N (N> 1) times according to the information and the number of N (N> 1) focal points, and the image capturing unit 150 controls the N (N) according to the autofocus control. > 1) two-dimensional images, and the image acquisition unit 165 acquires N (N> 1) two-dimensional image data corresponding to the photographed N (N> 1) two-dimensional images The focal two-dimensional image data acquisition cycle is preferably performed once for a unit time.

According to another exemplary embodiment of the present invention, when a 3D stereoscopic image conversion apparatus using the multifocal 2D image is provided in a video camera, N (N> 1) focal points determined by the focus information determiner 160 are provided. The focus controller 155 controls the autofocus N (N> 1) times according to the position information and the number of N (N> 1) focal points, and the image capturing unit 150 controls the N (N (N> 1)) according to the autofocus control. N> 1) two-dimensional images are captured, and the image acquisition unit 165 acquires N (N> 1) two-dimensional image data corresponding to the photographed N (N> 1) two-dimensional images. The multi-focus two-dimensional image data acquisition period is preferably repeated by the number of frames per second of the video camera.

For example, if the video camera captures 15 frames per second, the multi-focus two-dimensional image data acquisition cycle is preferably repeated 15 times per second, and if the number of focal points determined by the focus information determiner 160 is four The focus control unit 155 controls autofocus 60 times per second, the image capturing unit 150 captures 60 2D images per second, and the image obtaining unit 165 performs 60 2D image data per second. Will be obtained.

When the plurality of multi-focus 2D image data acquired through the image acquisition unit 165 is not converted into real-time 3D stereoscopic image data, the raw image storage unit 170 acquires one multi-focus 2D image data. The storage medium 183 stores N (N> 1) two-dimensional image data acquired during the cycle, the focal position information for each two-dimensional image data, and the focus control information in association with the multi-focus two-dimensional image data file. The multifocal 2D image data stored in the storage medium 183 is converted into 3D stereoscopic image data through the image conversion module 100.

Referring to FIG. 1, the 3D stereoscopic image converting apparatus using the multifocal 2D image is an input for reducing N (N> 1) 2D image data captured by the 2D image capturing module 145. An image reduction unit 105 and a focal length calculation unit that calculates a focal length of the N (N> 1) 2D image data based on the focus control information of the N (N> 1) 2D image data A focus overlapping processing unit 115 overlapping the 110 so that an object (or a focus area) corresponding to the focal position of the N (N> 1) two-dimensional image data is aligned along the line of sight, and N (N> 1) Depth value calculator 120 that calculates depth values for N (N> 1) two-dimensional image data in inverse proportion to the focal lengths for) two-dimensional image data, and N (N> 1) overlapping focal points. A depth image synthesizer 125 for synthesizing 2D image data according to the depth value, and the focal length and depth A disparity information checking unit for checking disparity processing information corresponding to the left eye and the right eye based on the left and right images; And a reduction ratio calculating unit 130 for calculating a reduction ratio for correcting horizontal distortion with respect to the left and right image data. It is characterized by comprising.

When N (N> 1) two-dimensional image data is acquired through the image acquisition unit 165 of the 2D image capturing module 145, the input image reduction unit 105 may convert the data of the image conversion module 100. In order to generate the left / right image data within a preset conversion time corresponding to the processing speed, the conversion size (or resolution) corresponding to the number N of 2D image data may be checked.

Here, if the data processing speed of the image conversion module 100 is constant, the shorter the conversion time, the smaller the transform size (or resolution), and the longer the conversion time, the larger the transform size (or resolution), The larger the number N of 2D image data, the smaller the transform size (or resolution), and the smaller the number N of 2D image data, the larger the transform size (or resolution).

For example, if the N (N> 1) two-dimensional image data input through the image acquisition unit 165 is one still image, the conversion time may be longer than several seconds, but the image acquisition unit 165 may be If the N (N> 1) two-dimensional image data input through the video, the conversion time should be short within a few to several tens of milliseconds corresponding to the number of frames per second.

When the conversion size (or resolution) is confirmed, the input image reduction unit 105 confirms the size (or resolution) of N (N> 1) two-dimensional image data input from the image acquisition unit 165. It is characterized in that the reduction to match the size (or resolution).

If the size (or resolution) of the N (N> 1) two-dimensional image data input from the image acquisition unit 165 is smaller than or equal to the identified conversion size (or resolution), the input image reduction unit ( 105 does not need to reduce the N (N> 1) two-dimensional image data, and the present invention is not limited thereto.

According to the exemplary embodiment of the present invention, the input image reduction unit 105 calculates a focal length for the N (N> 1) two-dimensional image data in association with the focal length calculation unit 110 and the depth value. A depth value is calculated for N (N> 1) two-dimensional image data in inverse proportion to the focal length in association with the calculation unit 120, and a left eye based on the focal length and depth value in conjunction with the parallax information checking unit. After checking the parallax angle corresponding to the right eye and the right eye, and calculating the reduction ratio for correcting the horizontal distortion of the generated left and right image data in connection with the reduction ratio calculating unit 130, the horizontal of the two-dimensional image data It is preferable to further reduce the size (or resolution) of the input N (N> 1) two-dimensional image data by further modifying the direction size according to the reduction ratio, and then the depth value image synthesizer 125 is configured to reduce the size. N (N> 1) Synthesizing 2D image data according to the depth value, and the left and right image generating unit 140 generates left eye image data and right eye image data by processing parallax on the synthesized 2D image data according to the parallax information. It is preferable.

The focal length calculating unit 110 is based on the focal position information and the focus number information determined by the focus information determining unit 160 included in the 2D image capturing module 145, and the image capturing unit 150 is N (N). N (N> 1) pieces of focus control information (eg, lens separation distance information for auto focus and the spaced apart information) identified by the focus control unit 155 in the process of controlling to capture> 1) two-dimensional images. The focal length between the lens and the object (or focal region) included in the N (N> 1) pieces of 2D image data is calculated using the refractive index information (or magnification information) of the lens.

Those skilled in the art to the present invention, by using the lens separation distance information for the auto focus at the primary lens position, and the refractive index information (or magnification information) of the spaced lens using the N (N> Since a process of calculating a focal length between an object (or a focus area) and a lens included in 1) 2D image data may be inferred, a detailed description thereof will be omitted for convenience.

When the focal lengths for the N (N> 1) two-dimensional image data obtained by the focal length calculation unit 110 are calculated, the focal overlap processing unit 115 performs the N (N> 1) two-dimensional images. Checking the focus position information corresponding to the data, it characterized in that the overlapping object (or focus area) corresponding to the focus position to be aligned in the line of sight.

When photographing a stationary subject, the subject does not move while the image capturing unit 150 captures N (N> 1) two-dimensional images for a unit of time by focus control of the focus controller 155. However, when the subject moves, the subject is controlled while the image capturing unit 150 captures N (N> 1) two-dimensional images for a unit of time by focus control of the focus controller 155. Since the focus overlapping processor 115 checks the focus position information corresponding to the N (N> 1) two-dimensional image data, the focus overlap processing unit 115 includes N (N> 1) two-dimensional image data included in the N (N> 1) two-dimensional image data. > 1) objects (or focal regions) are identified, and the identified N (N> 1) objects (or focal regions) overlap so as to be aligned along the line of sight.

According to an embodiment of the present invention, when the N (N> 1) two-dimensional image data includes N (N> 1) objects, the focus overlap processing unit 115 may have N (N> 1) objects. It is desirable to treat the boundaries for to overlap.

According to another exemplary embodiment of the present invention, when the N (N> 1) two-dimensional image data includes N (N> 1) focal regions, the focus overlap processing unit 115 may have N (N> 1). It is desirable to process the borders of the two focal regions so that they overlap.

When the focal lengths for the N (N> 1) two-dimensional image data obtained by the focal length calculator 110 are calculated, the depth value calculator 120 performs N (N> 1) two-dimensional image data. A depth value for N (N> 1) two-dimensional image data is calculated by arranging them in a distant order from the closest to the focal length. It is preferable that one depth value is set in the object (or the focus area) corresponding to.

According to an exemplary embodiment of the present invention, a depth value of two-dimensional image data having close focal lengths for N (N> 1) two-dimensional image data is set to '1', and the two-dimensional closest focal length is two-dimensional. Preferably, the depth value of the image data is set to '2', and the depth value of the two-dimensional image data having the furthest focal length is set to 'N'.

The focus overlapping unit 115 overlaps the focus positions of the N (N> 1) two-dimensional image data, and the depth value calculating unit 120 depths the N (N> 1) two-dimensional image data. When the value is calculated, the depth value image synthesizer 125 arranges the N (N> 1) two-dimensional image data of which the focus is overlapped according to the depth value, and the arranged N (N> 1) pieces And combining two-dimensional image data with the depth value.

According to the exemplary embodiment of the present invention, the depth value image synthesis unit 125 may include a 2D image corresponding to an mth (1 <= m <= N) depth value among N (N> 1) two-dimensional image data. Two-dimensional image data corresponding to the mth (0 <= m <N) object (or focus area) corresponding to the focal position of the data sharply and corresponding to the mth (1 <= m <= N) depth value Process another object (or focus area) in the transparent (or mth (1 <= m <= N) direction as the distance from the object (or focus area) increases), and the m (1 <= m <= N) In addition to the depth value, k (0 <= k <N, m! = K) corresponds to the focus position of the two-dimensional image data corresponding to the depth value k (0 <= k <N, m! = k) Clearly processes an object (or a focus area) and transparently clears another object (or focus area) area in the 2D image data corresponding to the k-th (0 <= k <N, m! = k) depth value. (Or transparency increases as you move away from the k (0 <= k <N, m! = K) object (or focus area)) The 2D image data corresponding to the mth (1 <= m <= N) depth value and the 2D image data corresponding to the kth (0 <= k <N, m! = K) depth value By combining, it is preferable to synthesize so that N (N> 1) objects (or focus areas) included in the N (N> 1) two-dimensional image data are clearly output.

The method of recognizing a distance through a left eye and a right eye includes a nonlinear function including a distance between the left eye and the right eye and a trigonometric function (or logarithmic function) for the distance from the left eye and the right eye to the focal point. The information checking unit has N (N> 1) focal lengths for the N (N> 1) two-dimensional image data according to the depth value calculated by the focal length calculating unit 110 and the binocular distance between the left and right eyes ( For example, the focus of N (N> 1) two-dimensional image data (or N (N> 1) two-dimensional image data based on the average binocular distance according to the average face size, which can be set by the key input unit 175). N (N> 1) disparity angle information of an object (or a focus area) corresponding to a position is calculated (or confirmed).

In addition, the parallax information confirming unit may zero-zero difference among N (N> 1) two-dimensional image data (or an object (or a focus area) corresponding to a focal position of N (N> 1) two-dimensional image data). Parallax) is characterized by identifying the parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focus position of the two-dimensional image data) to be processed.

According to an exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focus area) corresponding to a focal position of the 2D image data) includes N (N> 1) pieces of 2D image data ( Or among the N (N> 1) pieces of two-dimensional image data (or an object corresponding to the focal point), the focal length is located in the middle (for example, the middle of the closest and farthest focal lengths) Or two-dimensional image data (or an object (or focus area) corresponding to a focal position of the two-dimensional image data) in which the depth value is located in the middle (for example, the middle value of the smallest depth value and the largest depth value). It is desirable to confirm as a zero time difference object.

According to another exemplary embodiment of the present invention, the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) includes N (N> 1) two-dimensional image data. (Or two-dimensional image data (or two-dimensional image) having the shortest focal length or the smallest depth value (or an object corresponding to the focal position of N (N> 1) two-dimensional image data). Preferably, the object (or focus area) corresponding to the focus position of the data is identified as the zero parallax object.

According to another exemplary embodiment of the present invention, the parallax target 2D image data (or an object (or a focal region) corresponding to a focal position of the 2D image data) includes N (N> 1) 2D images. Among the image data (or an object (or focus area) corresponding to the focal position of N (N> 1) two-dimensional image data), the two-dimensional image data (or two having the largest focal length or the largest depth value) Preferably, the object (or focus area) corresponding to the focus position of the 3D image data is identified as the zero parallax object.

The parallax information confirming unit may be two-dimensional image data having a smaller focal length or a smaller depth than the two-dimensional image data (or an object corresponding to the focal position of the two-dimensional image data). (Or an object (or focus area) corresponding to the focal position of the 2D image data) and the two-dimensional image data (or an object corresponding to the focus position of the 2D image data) 2D image data (or 2) having a larger focal length or greater depth than the zero-parallel target 2D image data (or an object corresponding to a focal position of the 2D image data). An object (or second) corresponding to the focal position of the 2D image data (or a second object) corresponding to the focal position of the 2D image data is negative parallax. Point area)).

The disparity information checking unit may further include N (N> 1) for N (N> 1) two-dimensional image data (or an object (or focal region) corresponding to a focal position of N (N> 1) two-dimensional image data). 1) the parallax angle information and the focal length information are substituted into the trigonometric function (or logarithmic function) to the two-dimensional parallax target two-dimensional image data (or an object (or focal region) corresponding to the focal position of the two-dimensional image data). Verifying the parallax direction pixel movement information and the parallax direction pixel movement information of the parallax target 2D image data (or an object (or a focus area) corresponding to the focal position of the 2D image data). do.

According to the exemplary embodiment of the present invention, the parallax processing information checked by the parallax information confirming unit corresponds to the zero-disparity target 2D image data (or the focal position of the 2D image data included in the synthesized 2D image data). Object (or focal region)), at least one or more two-dimensional parallax image data (or an object (or focal region) corresponding to a focal position of the two-dimensional image data), bi-directional parallax direction pixel movement information, and at least one Preferably, the above parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focal position of the two-dimensional image data) and the parallax direction pixel movement information are included.

The parallax information checking unit checks the parallax target 2D image data (or an object (or a focus area) corresponding to the focus position of the 2D image data), and based on the parallax, at least one or more parallax targets 2 Two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) and bi-parallel direction pixel movement information are confirmed, and at least one or more parallax target two-dimensional image data (or two-dimensional image data) When the object (or focal region) corresponding to the focal position and the parallax direction pixel movement information are confirmed, the left and right image generating unit 140 performs bi-parallel target two-dimensional image data (the two-dimensional image data included in the synthesized two-dimensional image data). Or the pixel position corresponding to the object (or focal region) corresponding to the focal position of the 2D image data in the left and right directions according to the bi-parallel pixel movement information. And shift the pixel position corresponding to the parallax target 2D image data (or an object (or a focus area) corresponding to the focus position of the 2D image data) included in the synthesized 2D image data. The left eye image data and the right eye image data of the synthesized two-dimensional image data are generated by moving in the left direction and the right direction according to the pixel movement information.

According to the present invention, since the left eye image data and the right eye image data are generated based on the two-dimensional image data photographed by a single lens, the generated left eye image data and the right eye image data are not viewed by an actual human eye. The distortion ratio calculation unit 130 distorts the horizontal disparity information and the focal length information corresponding to the left eye image data and the right eye image data into a trigonometric function (or logarithmic function) and distorts them in the horizontal direction. And a reduction ratio for reducing and correcting the received image data.

According to the exemplary embodiment of the present invention, the reduction ratio calculator 130 disparity angle information corresponding to the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focus position of the two-dimensional image data). And a focal length information is substituted into a trigonometric function (or logarithmic function) to calculate a reduction ratio for the left eye image data and the right eye image data.

According to another exemplary embodiment of the present invention, the reduction ratio calculator 130 may include N (N> 1) two-dimensional image data (or two-dimensional image data) used to generate the left eye image data and the right eye image data. The parallax angle information and the focal length information corresponding to any two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) of the object corresponding to the focal position It is possible to calculate a reduction ratio for the left eye image data and the right eye image data by substituting a trigonometric function (or logarithmic function), and the present invention is not limited thereto.

Referring to FIG. 1, the 3D stereoscopic image conversion apparatus using the multifocal 2D image may be configured to reduce the left eye image data and the right eye image data generated by the image conversion module 100 according to the calculated reduction ratio. The image output unit 190 may be output through the screen output unit 180 or transmitted through the communication unit 195, or the left eye image data and the right eye image data generated by the image conversion module 100 may be one-to-one. And an image storage unit 193 that matches and stores the image storage unit 193 in the storage medium 183.

Left and right image data and right eye image data are generated by the left and right image generator 140 included in the image conversion module 100, and the left and right image data and the right eye image data are generated by the reduction ratio calculator 130. When the reduction ratio is calculated, the image output unit 190 reduces the left eye image data and the right eye image data according to the reduction ratio, and outputs the same through the screen output unit 180. The output unit 180 preferably includes a functional configuration for outputting a 3D stereoscopic image.

In addition, the image output unit 190 reduces the left eye image data and the right eye image data according to the reduction ratio and transmits them to another output device or a server (or device) on the communication network through the communication unit 195. It is done.

The image storage unit 193 may match left-eye image data and right-eye image data generated by the image conversion module 100 to one-to-one and store them in the storage medium 183. The left eye image data and the right eye image data may be reduced and stored in the storage medium 183.

2A and 2B illustrate a method of determining focus information for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

In detail, FIGS. 2A and 2B illustrate that N (N> 1) focal points of a subject are determined by the focus information determiner 160 included in the 3D stereoscopic image converting apparatus using the multifocal 2D image shown in FIG. 1. As a method of determining position information and focus number information, specifically, FIG. 2A illustrates N (N) when no identifiable object is included in the preview image data photographed and input through the image capturing unit 150. FIG. 2B illustrates a method of determining N (N> 1) focal position information and focus number information of> 1) focal regions, and FIG. 2B shows a preview image captured and input by the image capturing unit 150. When the identifiable object is included in the data, a method of determining N (N> 1) focal position information and focus number information for N (N> 1) objects is illustrated.

Those skilled in the art to which the present invention pertains, various methods for determining the focus information for the three-dimensional stereoscopic image conversion using the multi-focus two-dimensional image by referring to and / or modifying the drawings 2a and 2b Although the implementation method may be inferred, the present invention includes all the implementation methods inferred above, and the technical features are not limited only to the implementation methods shown in FIGS. 2A and 2B.

2A and 2B, the focus information determiner 160 automatically recognizes N (N> 1) objects included in the preview image data through pattern recognition (or image recognition), or By manually selecting N (N> 1) objects included in the preview image data in association with the key input unit 175, N (N> 1) objects (eg, human, animal) are included in the preview image data. , Object such as an object including a boundary distinguishing from a background (or other object) is included.

Referring to FIG. 2A, in which the identifiable object is not included in the preview image data photographed and input by the image capturing unit 150, the focus information determiner 160 stores the preview image data as N (N>). 1) dividing into focus areas, and determining the center positions of the N (N> 1) focus areas as N (N> 1) focus points, and the number of N (N> 1) focus points. Is determined as N (N> 1) focal number.

Here, the N (N> 1) pieces of focal number information is the maximum number of shots that the image capturing unit 150 can capture for a unit time, or the focus control unit 155 of the image capturing unit 150 for a unit time. At least one maximum control number for controlling the focus, or at least one maximum number of image data acquisition that the image acquisition unit 165 can obtain two-dimensional image data from the image capture unit 150 for a unit time It is preferable to set smaller than the limit focal number.

According to an exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the vertical direction as shown in FIG. 2A. As described above, it is preferable to determine the center positions of the N (N> 1) focal regions divided in the vertical direction as the N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the left and right directions, and the N divided in the left and right directions as described above. It is preferable to determine the center positions of the (N> 1) focal regions as N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions according to a grid shape, and divides the preview image data into a grid shape as described above. It is preferable to determine the center position of the N (N> 1) focusing areas as N (N> 1) focusing positions.

Referring to FIG. 2B including an object that can be identified in the preview image data photographed and input by the image capturing unit 150, the focus information determiner 160 includes N (N) included in the preview image data. And N (N> 1) focal positions, and determine the number of N (N> 1) focal positions as N (N> 1) focal points. It is characterized by.

Here, the N (N> 1) pieces of focal number information is the maximum number of shots that the image capturing unit 150 can take for a unit time (for example, 50 ms, etc.) or the focus control unit 155 takes the image for a unit time. Maximum number of control times for controlling the focus of the photographing unit 150 or maximum number of image data acquisition units for obtaining the 2D image data from the image photographing unit 150 by the image obtaining unit 165 for a unit time. It is preferably set smaller than the limit focal number including at least one.

According to an exemplary embodiment of the present disclosure, when the number of focus positions (or the number of focal points) included in the preview image data is greater than the limit focal number, the focus information photographing unit may include the preview image among the objects included in the preview image data. It is preferable to determine N (N> 1) focal positions for objects within the above limit focal number among objects at the center position (or top, bottom, left, or right) of the viewing image data.

According to another exemplary embodiment of the present invention, when the number of focus positions (or the number of focus points) included in the preview image data is less than the limit focus number, the focus information photographing unit includes N (N) included in the preview image data. It is desirable to determine N (N> 1) focal positions for> 1) objects.

3A and 3B illustrate a focal length calculation method for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

In detail, FIGS. 3A and 3B show that the focal length calculation unit 110 included in the 3D stereoscopic image converting apparatus using the multifocal 2D image shown in FIG. 1 is applied to N (N> 1) 2D image data. FIG. 3A illustrates a method of calculating a focal length for the N (N> 1) two-dimensional image data based on the focus control information. More specifically, FIG. 3A is photographed and input through the image capturing unit 150. When the identifiable object is not included in the preview image data, a method of calculating a focal length for N (N> 1) two-dimensional image data is illustrated, and FIG. 3B is provided through the image capturing unit 150. When a identifiable object is included in the preview image data photographed and input, a method of calculating a focal length for N (N> 1) two-dimensional image data is illustrated.

Those skilled in the art to which the present invention pertains, various reference to the focal length calculation method for the three-dimensional stereoscopic image conversion using the multi-focal two-dimensional image by referring to and / or modifying the drawings 3a and 3b Although the implementation method may be inferred, the present invention includes all the implementation methods inferred above, and the technical features are not limited only to the implementation methods shown in FIGS. 3A and 3B.

Referring to FIGS. 3A and 3B, the focal length calculator 110 determines that the image capturing unit 150 is N (N> 1) based on the focal position information and the focal number information determined by the focus information determiner 160. N (N> 1) pieces of focus control information (for example, lens separation distance information for auto focusing and the spaced apart lenses) identified by the focus control unit 155 in the process of controlling to take two 2D images. The focal length between the lens and the object (or focal region) included in the N (N> 1) pieces of 2D image data is calculated using refractive index information (or magnification information).

Referring to FIG. 3a in which the identifiable object is not included in the preview image data photographed and input by the image capturing unit 150, N (N> 1) focus regions are vertically upper and lower as shown in FIG. 2a. In the case of dividing in the direction, the focal length calculator 110 determines the focus control information (eg, lens separation distance information for auto focus and the refractive index information of the spaced apart lens) identified in the process of auto focusing each focus area. Magnification information), etc.), to calculate a focal length for each two-dimensional image data, and to calculate the focal length of the bottom area is the closest and the focal length of the top area is the farthest. It is done.

At this time, the focus area that is autofocused in the N (N> 1) two-dimensional image data includes clear image data, and includes blurry image data as the distance from the focus area increases.

Referring to FIG. 3B including an object that can be identified in the preview image data photographed and input by the image capturing unit 150, as shown in FIG. 2B, N (N> 1) pieces of the preview image data are shown. When the object is identified, the focal length calculator 110 determines the focus control information (eg, lens separation distance information for auto focus and the refractive index information of the spaced apart lens) identified in the process of auto focusing each object. Magnification information), etc.) to calculate a focal length for each two-dimensional image data, wherein the object having the shortest lens separation distance is the closest, and the object having the long lens separation distance is calculated to be the farthest. It features.

In this case, the autofocused object in the N (N> 1) two-dimensional image data includes clear image data and includes blurry image data as it moves away from the focus area.

4A and 4B illustrate a focus overlapping process for converting a 3D stereoscopic image using a multifocal 2D image according to an exemplary embodiment of the present invention.

In detail, FIGS. 4A and 4B show that the focus overlap processing unit 115 included in the 3D stereoscopic image converting apparatus using the multifocal 2D image shown in FIG. A method of overlapping an object (or a focus area) corresponding to a focal position so as to be aligned according to a line of sight, specifically, FIG. 4A is identified in the preview image data captured and input through the image capturing unit 150. When the possible objects are not included, a method of overlapping the focus areas corresponding to the focal positions of the N (N> 1) two-dimensional image data to be aligned along the line of sight is illustrated, and FIG. 4B illustrates the image capturing unit ( When the identifiable object is included in the preview image data captured and input through 150, an object corresponding to the focal position of the N (N> 1) two-dimensional image data is determined according to the visual direction. How to overlap so that illustrates a.

Those skilled in the art to which the present invention pertains, various implementations of the focus overlap process for the three-dimensional stereoscopic image conversion using the multi-focus two-dimensional image by referring to and / or modified with reference to the drawings 4a and 4b The method may be inferred, but the present invention includes all the inferred implementation methods, and the technical features are not limited to the implementation method shown in FIGS. 4A and 4B.

4A and 4B, the focus overlapping processor 115 checks focus position information corresponding to the N (N> 1) two-dimensional image data, and identifies an object (or a focus area) corresponding to the focus position. ) Is superimposed so as to be aligned along the line of sight.

When photographing a stationary subject, the subject does not move while the image capturing unit 150 captures N (N> 1) two-dimensional images for a unit of time by focus control of the focus controller 155. However, when the subject moves, the subject moves while the image capturing unit 150 captures N (N> 1) two-dimensional images for a unit time by focus control of the focus controller 155. Since the focus overlapping processing unit 115 checks the focus position information corresponding to the N (N> 1) two-dimensional image data, the focus overlap processing unit 115 includes N (N> 1) two-dimensional image data included in the N (N> 1) two-dimensional image data. > 1) objects (or focal regions) are identified, and the identified N (N> 1) objects (or focal regions) overlap so as to be aligned along the line of sight.

Referring to FIG. 4A, in which the identifiable object is not included in the preview image data photographed and input through the image capturing unit 150, as illustrated in FIG. 2A, the N (N> 1) two-dimensional images are included. When N (N> 1) focus areas are included in the data, the focus overlapping processing unit 115 may process N (N> 1) focus area boundaries to overlap.

Referring to FIG. 4B, in which the identifiable object is included in the preview image data photographed and input by the image capturing unit 150, N (N> 1) objects are included in N (N> 1) two-dimensional image data. If is included, the focus overlapping processor 115 is characterized in that the processing for overlapping the boundaries for the N (N> 1) objects.

5A and 5B illustrate a depth value calculation process for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

In detail, FIGS. 5A and 5B show that the depth calculator 120 included in the 3D stereoscopic image converting apparatus using the multifocal 2D image shown in FIG. 1 applies N (N> 1) two-dimensional image data. FIG. 5A illustrates a method of calculating depth values of N (N> 1) two-dimensional image data in inverse proportion to a focal length. Specifically, FIG. 5A is a preview captured by the image capturing unit 150. When the identifiable object is not included in the image data, depth values of N (N> 1) two-dimensional image data are inversely proportional to the focal length of the focal region included in the N (N> 1) two-dimensional image data. FIG. 5B is a diagram illustrating a calculation method, and FIG. 5B is included in N (N> 1) two-dimensional image data when an identifiable object is included in the preview image data captured and input by the image capturing unit 150. The focal length of the object Inversely proportional to illustrate a method of calculating a depth value for the N (N> 1) of the two-dimensional image data.

Those skilled in the art to which the present invention pertains, various methods for depth value calculation process for 3D stereoscopic image conversion using the multi-focus 2D image by referring to and / or modifying the drawings 5a and 5b. Although the implementation method may be inferred, the present invention includes all the implementation methods inferred, and the technical features are not limited only to the implementation methods shown in FIGS. 5A and 5B.

5A and 5B, the depth calculator 120 focuses on N (N> 1) two-dimensional image data (or an object (or a focus area) corresponding to a focus position of the two-dimensional image data). And calculating one depth value in one piece of 2D image data (or an object (or a focus area) corresponding to a focus position of the 2D image data) in inverse proportion to the distance.

Referring to FIG. 5A, in which the identifiable object is not included in the preview image data photographed and input by the image capturing unit 150, as illustrated in FIG. 2A, the N (N> 1) two-dimensional images are included. When the data includes N (N> 1) focal regions, the depth calculator 120 determines a depth value of 2D image data having a near focal length among the N (N> 1) focal regions as '1'. The depth value of the 2D image data having the second closest focal length is set to '2', and the depth value of the 2D image data having the furthest focal length is set to 'N'. .

Referring to FIG. 5B, in which the identifiable object is included in the preview image data captured and input by the image capturing unit 150, N (N> 1) objects are included in N (N> 1) two-dimensional image data. Is included, the depth calculator 120 sets the depth value of the two-dimensional image data having the closest focal length among the N (N> 1) objects to '1' and the second closest to the focal length. The depth value of the 2D image data is set to '2', and the depth value of the 2D image data having the furthest focal length is set to 'N'.

6 is a diagram illustrating a distance recognition method for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

In more detail, in FIG. 6, the parallax information confirming unit included in the 3D stereoscopic image converting apparatus using the multifocal 2D image shown in FIG. 1 recognizes a distance corresponding to the left and right eyes based on the focal length and the depth value. As shown in the drawings, a person of ordinary skill in the art to which the present invention pertains can refer to and / or modify the drawing 6 and distance recognition method for 3D stereoscopic image conversion using the multifocal 2D image. Various embodiments of the present invention can be inferred, but the present invention includes all the implementation methods inferred above, and the technical features are not limited only to the embodiment shown in FIG.

A method of recognizing a distance through a left eye and a right eye includes a nonlinear function including a distance between the left eye and the right eye and a trigonometric function (or logarithmic function) for the distance from the left eye and the right eye to the focal point. Referring to FIG. 6, a human subject within a range capable of focusing according to each person's vision (or corrected vision) (referred to as “minimum distance” in FIG. 6) recognizes a distance according to a trigonometric function, and focuses on the subject. The subject in a difficult range (eg, a distance farther than the minimum distance) is characterized by recognizing the distance according to the logarithmic function.

On the other hand, a digital camera (or video camera) equipped with a three-dimensional stereoscopic image conversion apparatus using a multi-focus two-dimensional image shown in FIG. 1 sharply even a subject existing in the logarithmic function area according to lens performance (or magnification). You can focus.

When the focal length is calculated by the focal length calculator 110 for N (N> 1) two-dimensional image data (or an object (or a focal region) corresponding to a focal position of the two-dimensional image data), the parallax information The identification unit may determine whether the focal length is within the minimum distance recognized by the trigonometric function or outside the minimum distance recognized by the logarithmic function, wherein the minimum distance is 3 converted from the multi-focus 2D image. By using an average value of a group of viewers viewing a 3D stereoscopic image, directly setting the minimum distance through the key input unit 175, or inputting a visual acuity (or corrected vision) value through the key input unit 175. Preferably, the minimum distance corresponding to the visual acuity (or correction visual acuity) is automatically set.

According to an embodiment of the present invention, the first to n-th (n = 2,3, ..., N) two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) When the focal length is outside the minimum distance (eg, a logarithmic region), the parallax information checking unit disparages the corresponding 2D image data (or an object (or a focal region) corresponding to the focal position of the 2D image data) It is preferable to exclude from the processing range, and when the N (N> 1) two-dimensional image data is enlarged as a whole (for example, by using a telephoto lens), the starting position of the minimum distance is determined. It is possible to set the subject to be located within the minimum distance by virtually setting a position away from the lens by the subject.

In the N (N> 1) two-dimensional image data (or an object corresponding to a focal position of the two-dimensional image data) in which the focal length is located within the minimum distance, the parallax information checking unit is arranged in the focal point. N (N> 1) focal lengths for the N (N> 1) two-dimensional image data according to the depth value calculated by the distance calculating unit 110 and the binocular distance between the left and right eyes (eg, average face size) An object corresponding to the focal position of N (N> 1) two-dimensional image data (or N (N> 1) two-dimensional image data) based on an average binocular distance according to (Or focal region)) N (N> 1) pieces of parallax angle information are calculated (or confirmed).

7 illustrates a parallax processing method for 3D stereoscopic image conversion using a multifocal 2D image according to an exemplary embodiment of the present invention.

In more detail, in FIG. 7, the parallax information checking unit included in the 3D stereoscopic image converting apparatus using the multifocal 2D image shown in FIG. 1 corresponds to the left and right eyes based on the focal length and depth values. As a method for generating a digital signal, a person having ordinary skill in the art to which the present invention pertains can perform parallax for converting a 3D stereoscopic image using the multifocal 2D image by referring to and / or modifying the drawing 7. Various implementation methods for the processing method may be inferred, but the present invention includes all the implementation methods inferred above, and the technical features are not limited only to the implementation method shown in FIG.

Through the distance recognition method shown in FIG. 6, N (N> 1) two-dimensional image data (or an object corresponding to a focal position of two-dimensional image data) in which the focal length is located within the minimum distance. Once the parallax information for) is calculated (or confirmed),

The parallax information checking unit performs zero parallax among N (N> 1) two-dimensional image data (or an object (or a focus area) corresponding to a focal position of N (N> 1) two-dimensional image data). It is characterized by identifying the zero-parallel target two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) to be processed.

According to an exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focus area) corresponding to a focal position of the 2D image data) includes N (N> 1) pieces of 2D image data ( Or among the N (N> 1) pieces of two-dimensional image data (or an object corresponding to the focal point), the focal length is located in the middle (for example, the middle of the closest and farthest focal lengths) Or two-dimensional image data (or an object corresponding to a focal position of the two-dimensional image data) whose depth value is located in the middle (eg, the middle of the smallest depth value and the largest depth value). It is preferable to confirm with the said zero parallax object.

According to another exemplary embodiment of the present invention, the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) includes N (N> 1) two-dimensional image data. (Or two-dimensional image data (or two-dimensional image) having the shortest focal length or the smallest depth value (or an object corresponding to the focal position of N (N> 1) two-dimensional image data). Preferably, the object (or focus area) corresponding to the focus position of the data is identified as the zero parallax object.

According to another exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focal region) corresponding to a focal position of the 2D image data) includes N (N> 1) 2D images. Two-dimensional image data (or two-dimensional) having the longest focal length or the largest depth value among the data (or objects (or focal regions) corresponding to the focal positions of N (N> 1) two-dimensional image data) Preferably, the object (or focus area) corresponding to the focus position of the image data is identified as the zero parallax object.

The parallax information confirming unit may be two-dimensional image data having a smaller focal length or a smaller depth than the two-dimensional image data (or an object corresponding to the focal position of the two-dimensional image data). (Or an object (or focus area) corresponding to the focal position of the 2D image data) is a positive parallex target 2D image data (or an object (or focus area) corresponding to the focal position of the 2D image data). 2D image data (or a larger focal length or greater depth than the zero-parallel target 2D image data (or an object (or a focus area) corresponding to the focal position of the 2D image data)) An object corresponding to a focal position of two-dimensional image data (or an object corresponding to a focal position of two-dimensional image data) (or a focus area) Characterized in that a check point area)).

The disparity information checking unit may further include N (N> 1) for N (N> 1) two-dimensional image data (or an object (or focal region) corresponding to a focal position of N (N> 1) two-dimensional image data). 1) disparity angle information and focal length information are substituted into a trigonometric function (or logarithmic function), and as shown in FIG. 7, the two-dimensional parallax object 2D image data (or the focal position of the 2D image data) Bilateral parallax pixel movement information for an object (or focal region) and parallax pixel for the parallax target 2D image data (or an object (or focal region) corresponding to a focal position of the 2D image data) Characterize the movement information.

8A and 8B illustrate a 3D stereoscopic image conversion process using a multifocal 2D image according to an exemplary embodiment of the present invention.

In more detail, FIGS. 8A and 8B illustrate N (N> 1) pieces of focus information in the 3D stereoscopic image conversion apparatus using the multi-focus 2D image shown in FIG. 1, and the determined N (N> 1) Acquire N (N> 1) 2D image data based on the focus information, and convert the size (or resolution) of the obtained N (N> 1) 2D image data into a predetermined transform size (or resolution). 2), calculate a focal length for the N (N> 1) two-dimensional image data, process a focal overlap, and calculate a depth value inversely proportional to the focal length. A method of synthesizing dimensional image data, checking disparity processing information on the synthesized 2D image data, and generating left eye image data and right eye image data based on the disparity processing information.

Those skilled in the art to which the present invention pertains may refer to and / or modify the drawings 8a and 8b to infer various implementation methods for the 3D stereoscopic image conversion process using the multifocal 2D image. As will be appreciated, the present invention includes all the inferred implementation methods, and the technical features are not limited to the implementation method shown in FIGS. 8A and 8B.

For example, in the process of synthesizing a multi-focus two-dimensional image in a three-dimensional stereoscopic image conversion process using the multi-focus two-dimensional image in FIGS. 8A and 8B, the focal lengths for (N> 1) two-dimensional image data are calculated. Although a process of focus overlapping and calculating a depth value that is inversely proportional to the focal length is shown, the process of synthesizing the multifocal two-dimensional image is not necessarily processed according to a previous order. The order of some of the processes may be changed (or omitted) according to the intention of those skilled in the art, and the present invention is not limited thereto.

Alternatively, the focal length is reduced after reducing the size (or resolution) of the (N> 1) pieces of 2D image data obtained in a 3D stereoscopic image conversion process using a multifocal 2D image in FIGS. 8A and 8B. And calculating the parallax processing information after calculating (N> 1) two-dimensional image data by calculating a depth value and a depth value, but the size of the obtained (N> 1) two-dimensional image data. (Or the resolution) may be reduced after the focal length and the parallax processing information are confirmed and before synthesizing the (N> 1) two-dimensional image data, and the present invention is not limited thereto.

8A and 8B, the focus information determiner 160 included in the 3D stereoscopic image converting apparatus using the multifocal 2D image illustrated in FIG. 1 is photographed by the image capturing unit 150. N (N> 1) objects included in the input preview image data are automatically recognized through pattern recognition (or image recognition), or N included in the preview image data in connection with the key input unit 175. By manually selecting (N> 1) objects, the preview image data includes boundaries that are distinguished from N (N> 1) objects (e.g., a background (or other object, such as a person, an animal, or an object)). By determining whether a subject is included, N (N> 1) focal points and the number of foci of the N (N> 1) objects (or focal regions) are determined as shown in FIG. 2A or 2B. (800).

According to an exemplary embodiment of the present invention, when the identifiable object is not included in the preview image data, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions. Preferably, the center positions of the N (N> 1) focal regions are determined as N (N> 1) focal positions, and the number of N (N> 1) focal positions is N (N> 1) focal points. It is preferable to determine the number.

Here, the N (N> 1) pieces of focal number information is the maximum number of shots that the image capturing unit 150 can capture for a unit time, or the focus control unit 155 of the image capturing unit 150 for a unit time. At least one maximum control number for controlling the focus, or at least one maximum number of image data acquisition that the image acquisition unit 165 can obtain two-dimensional image data from the image capture unit 150 for a unit time It is preferable to set smaller than the limit focal number.

According to the exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the vertical direction as shown in FIGS. 8A and 8BA. As described above, it is preferable to determine the center positions of the N (N> 1) focal regions divided in the vertical direction as the N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the left and right directions, and the N divided in the left and right directions as described above. It is preferable to determine the center positions of the (N> 1) focal regions as N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions according to a grid shape, and divides the preview image data into a grid shape as described above. It is preferable to determine the center position of the N (N> 1) focusing areas as N (N> 1) focusing positions.

According to another exemplary embodiment of the present invention, when the identifiable object is included in the preview image data, the focus information determiner 160 may include the centers of N (N> 1) objects included in the preview image data. Preferably, the position is determined to be N (N> 1) focal positions, and the number of N (N> 1) focal positions is preferably determined to be N (N> 1) focal positions.

In this case, the N (N> 1) pieces of focus information is the maximum number of shots that the image capturing unit 150 can take for a unit time (for example, 50 ms, etc.) or the focus control unit 155 takes the image for a unit time. Maximum number of control times for controlling the focus of the photographing unit 150 or maximum number of image data acquisition units for obtaining the 2D image data from the image photographing unit 150 by the image obtaining unit 165 for a unit time. It is preferably set smaller than the limit focal number including at least one.

According to an exemplary embodiment of the present disclosure, when the number of focus positions (or the number of focal points) included in the preview image data is greater than the limit focal number, the focus information photographing unit may include the preview image among the objects included in the preview image data. It is preferable to determine N (N> 1) focal positions for objects within the above limit focal number among objects at the center position (or top, bottom, left, or right) of the viewing image data.

According to another exemplary embodiment of the present invention, when the number of focus positions (or the number of focus points) included in the preview image data is less than the limit focus number, the focus information photographing unit includes N (N) included in the preview image data. It is desirable to determine N (N> 1) focal positions for> 1) objects.

When the N (N> 1) focal position information and the N (N> 1) focal number are determined by the focus information determiner 160, the focus controller 155 may focus on the image capturing unit 150. Control to autofocus to a first focus position among the N (N> 1) focus positions to capture a first 2D image through the image capturing unit 150 (805), and the image obtaining unit 165 ) Acquires the first two-dimensional image data for the first focus position (810), and correspondingly, the focus controller 155 confirms the first focus control information for the first auto focus (815), In the process, until N (N> 1) two-dimensional image data for N (N> 1) focal positions are obtained and N (N> 1) focal position focus control information is confirmed (825), In operation 820, the n th (n = 2,3, ..., N) focal point is repeatedly performed.

When the N (N> 1) two-dimensional image data is obtained through the process, and the N (N> 1) focal position focus control information is confirmed (825), the input image reduction unit 105 performs the two-dimensional image. Check the conversion size (or resolution) corresponding to the number N of image data, and compare the size (or resolution) of the input N (N> 1) two-dimensional image data with the checked size (or resolution). In operation 830, it is checked whether to reduce or reduce the size (or resolution) of the N (N> 1) two-dimensional image data.

According to the exemplary embodiment of the present invention, the input image reduction unit 105 may convert the left / right image data within a preset conversion time (or corresponding to the data processing speed of the image conversion module 100). Resolution).

Here, if the data processing speed of the image conversion module 100 is constant, the shorter the conversion time, the smaller the transform size (or resolution), and the longer the conversion time, the larger the transform size (or resolution), The larger the number N of 2D image data, the smaller the transform size (or resolution), and the smaller the number N of 2D image data, the larger the transform size (or resolution).

For example, if the N (N> 1) two-dimensional image data input through the image acquisition unit 165 is one still image, the conversion time may be longer than several seconds, but the image acquisition unit 165 may be If the N (N> 1) two-dimensional image data input through the video, the conversion time should be short within a few to several tens of milliseconds corresponding to the number of frames per second.

When the conversion size (or resolution) is confirmed, the input image reduction unit 105 confirms the size (or resolution) of N (N> 1) two-dimensional image data input from the image acquisition unit 165. The size (or resolution) of the N (N> 1) two-dimensional image data input from the image acquisition unit 165 is determined as the transformed size (or Resolution), the input image reduction unit 105 may not reduce the N (N> 1) two-dimensional image data.

If the result of the check indicates that the size (or resolution) of the N (N> 1) two-dimensional image data needs to be reduced and converted (835), the input image reduction unit 105 may obtain the obtained N (N> 1). The size (or resolution) of two pieces of 2D image data is reduced and converted according to the conversion size (or resolution) (840).

The focal length calculator 110 uses the identified N (N> 1) pieces of focus control information (eg, lens separation distance information for auto focus, refractive index information (or magnification information), etc. of the spaced apart lens). As shown in FIG. 3A or 3B, a focal length between an object (or a focal region) and a lens included in the N (N> 1) two-dimensional image data is calculated (845).

If the focal lengths for the N (N> 1) two-dimensional image data are calculated (850), the focus overlap processing unit 115 provides the focal position information corresponding to the N (N> 1) two-dimensional image data. In operation 855, the object (or the focus area) corresponding to the focus position is aligned in line of sight and overlapped as shown in FIG. 4A or 4B.

If the N (N> 1) two-dimensional image data are superimposed (860), the depth calculator 120 performs the order in which the focal lengths for the N (N> 1) two-dimensional image data are close to each other in a distant order. Arranged to calculate depth values for N (N> 1) two-dimensional image data (865), where one two-dimensional image data (or an object corresponding to a focal position of the two-dimensional image data (or a focus area)) Preferably, one depth value is set at.

According to an exemplary embodiment of the present invention, a depth value of two-dimensional image data having close focal lengths for N (N> 1) two-dimensional image data is set to '1', and the two-dimensional closest focal length is two-dimensional. Preferably, the depth value of the image data is set to '2', and the depth value of the two-dimensional image data having the furthest focal length is set to 'N'.

If depth values of the N (N> 1) two-dimensional image data are calculated (870), the depth value image synthesis unit 125 performs N (N> 1) two-dimensional image data of which the focus is overlapped. Are arranged according to the depth value, and the arranged N (N> 1) two-dimensional image data are synthesized to the depth value to generate short-circuit two-dimensional image data (875).

According to the exemplary embodiment of the present invention, the depth value image synthesis unit 125 may include a 2D image corresponding to an mth (1 <= m <= N) depth value among N (N> 1) two-dimensional image data. Two-dimensional image data corresponding to the mth (0 <= m <N) object (or focus area) corresponding to the focal position of the data sharply and corresponding to the mth (1 <= m <= N) depth value Process another object (or focus area) in the transparent (or mth (1 <= m <= N) direction as the distance from the object (or focus area) increases), and the m (1 <= m <= N) In addition to the depth value, k (0 <= k <N, m! = K) corresponds to the focus position of the two-dimensional image data corresponding to the depth value k (0 <= k <N, m! = k) Clearly processes an object (or a focus area) and transparently clears another object (or focus area) area in the 2D image data corresponding to the k-th (0 <= k <N, m! = k) depth value. (Or transparency increases as you move away from the k (0 <= k <N, m! = K) object (or focus area)) The 2D image data corresponding to the mth (1 <= m <= N) depth value and the 2D image data corresponding to the kth (0 <= k <N, m! = K) depth value By combining, it is preferable to synthesize so that N (N> 1) objects (or focus areas) included in the N (N> 1) two-dimensional image data are clearly output.

Subsequently, the parallax information checking unit checks the parallax processing information corresponding to the left and right eyes according to the distance recognition method through the left and right eyes of a human (880).

According to the exemplary embodiment of the present invention, the parallax information checking unit has N (N> 1) focal lengths for N (N> 1) two-dimensional image data according to the depth value calculated by the focal length calculating unit 110. And N (N> 1) two-dimensional image data (or N (N) based on the binocular distance between the left eye and the right eye (for example, the average binocular distance according to the average face size, which can be set through the key input unit 175). It is preferable to calculate (or confirm) N (N> 1) parallax angle information for an object (or a focus area) corresponding to a focus position of> 1) two-dimensional image data.

Also, as shown in FIG. 7, the parallax information checking unit (or focal point) corresponds to a focal position of N (N> 1) two-dimensional image data (or N (N> 1) two-dimensional image data). Region)), it is preferable to identify the zero-parallel target two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) to be processed by zero parallax.

According to an exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focus area) corresponding to a focal position of the 2D image data) includes N (N> 1) pieces of 2D image data ( Or among the N (N> 1) pieces of two-dimensional image data (or an object corresponding to the focal point), the focal length is located in the middle (for example, the middle of the closest and farthest focal lengths) Or two-dimensional image data (or an object (or focus area) corresponding to a focal position of the two-dimensional image data) in which the depth value is located in the middle (for example, the middle value of the smallest depth value and the largest depth value). It is desirable to confirm as a zero time difference object.

According to another exemplary embodiment of the present invention, the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) includes N (N> 1) two-dimensional image data. (Or two-dimensional image data (or two-dimensional image) having the shortest focal length or the smallest depth value (or an object corresponding to the focal position of N (N> 1) two-dimensional image data). Preferably, the object (or focus area) corresponding to the focus position of the data is identified as the zero parallax object.

According to another exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focal region) corresponding to a focal position of the 2D image data) includes N (N> 1) 2D images. Two-dimensional image data (or two-dimensional) having the longest focal length or the largest depth value among the data (or objects (or focal regions) corresponding to the focal positions of N (N> 1) two-dimensional image data) Preferably, the object (or focus area) corresponding to the focus position of the image data is identified as the zero parallax object.

Also, as shown in FIG. 7, the disparity information checking unit has a focal length that is closer than the zero parallax target two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data), or Two-dimensional image data having a small depth value (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) corresponds to the two-dimensional image data (or the focal position of the two-dimensional image data) subject to positive parallex Object (or focus area), and the focal length is farther than the zero parallax target 2D image data (or the object (or focus area) corresponding to the focus position of the 2D image data), or the depth value is Large two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) is subjected to a negative parallex target two-dimensional image data (or second of two-dimensional image data). To make an object (or the focal region)) corresponding to the position are preferred.

Also, as shown in FIG. 7, the disparity information checking unit (or focus area) corresponds to a focal position of N (N> 1) two-dimensional image data (or N (N> 1) two-dimensional image data). N (N> 1) disparity angle information and focal length information for)) are substituted into a trigonometric function (or logarithmic function) to correspond to the focal position of the two-dimensional parallax object 2D image data (or 2D image data). Bilateral parallax pixel movement information for an object (or focal region) and parallax pixel for the parallax target 2D image data (or an object (or focal region) corresponding to a focal position of the 2D image data) It is desirable to confirm the movement information.

According to the exemplary embodiment of the present invention, the parallax processing information checked by the parallax information confirming unit corresponds to the zero-disparity target 2D image data (or the focal position of the 2D image data included in the synthesized 2D image data). Object (or focal region)), at least one or more two-dimensional parallax image data (or an object (or focal region) corresponding to a focal position of the two-dimensional image data), bi-directional parallax direction pixel movement information, and at least one Preferably, the above parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focal position of the two-dimensional image data) and the parallax direction pixel movement information are included.

If the parallax processing information corresponding to the left eye and the right eye is confirmed (885), the left and right image generating unit 140, as shown in FIG. 7, the two-dimensional parallax object 2D included in the synthesized two-dimensional image data The pixel position corresponding to the image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) is moved in the left and right directions according to the bi-directional pixel movement information, and the synthesized two-dimensional Pixel positions corresponding to the parallax target 2D image data (or an object (or a focus area) corresponding to the focal position of the 2D image data) included in the image data may be left and right according to the parallax direction pixel movement information. In operation 890, left eye image data and right eye image data of the synthesized two-dimensional image data are generated.

If left eye image data and right eye image data for the synthesized 2D image data are generated (892), the reduction ratio calculator 130 may disparity angle information and focal length corresponding to the left eye image data and the right eye image data. The reduction ratio for reducing and correcting the image data distorted in the horizontal direction is calculated by substituting the information into a trigonometric function (or logarithmic function) (894).

According to the exemplary embodiment of the present invention, the reduction ratio calculator 130 disparity angle information corresponding to the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focus position of the two-dimensional image data). And a focal length information is substituted into a trigonometric function (or logarithmic function) to calculate a reduction ratio for the left eye image data and the right eye image data.

According to another exemplary embodiment of the present invention, the reduction ratio calculator 130 may use N (N> 1) two-dimensional image data (or two-dimensional image data) used to generate the left eye image data and the right eye image data. Parallax angle information and focal length information corresponding to any two-dimensional image data (or an object (or a focal region) corresponding to a focal position of the two-dimensional image data) of the object corresponding to the focal position of It is possible to calculate a reduction ratio for the left eye image data and the right eye image data by substituting a trigonometric function (or a logarithmic function), and the present invention is not limited thereto.

If a reduction ratio of the left eye image data and the right eye image data is calculated (896), the image output unit 190 reduces the generated left eye image data and the right eye image data according to the calculated reduction ratio and outputs the screen. Output through the unit 180 or through the communication unit 195, or the image storage unit 193 is one-to-one matching the generated left eye image data and the right eye image data is stored in the storage medium 183 (898).

9A and 9B illustrate a 3D stereoscopic image conversion process using a multifocal 2D image according to another exemplary embodiment of the present invention.

In more detail, FIGS. 9A and 9B illustrate the N (N> 1) pieces of focus information in the 3D stereoscopic image conversion apparatus using the multifocal 2D image shown in FIG. 1, and the determined N (N> 1) Acquiring N (N> 1) two-dimensional image data based on the focus information, calculating a focal length for the N (N> 1) two-dimensional image data, and depth inversely proportional to the focal length Calculates the parallax processing information, calculates the horizontal reduction ratio according to the focal length and the parallax processing information, and calculates the size (or resolution) of the obtained N (N> 1) two-dimensional image data. After reducing to a predetermined conversion size (or resolution), the image is reduced by applying the reduction ratio, and focusing is performed on the N (N> 1) two-dimensional image data, and a depth value inversely proportional to the focal length is calculated. N (N> 1) two-dimensional image data And a diagram showing an exemplary method for generating the left eye image data and right eye image data based on the differential management information.

Those skilled in the art to which the present invention pertains refer to the conversion process shown in FIGS. 8A and 8B and the conversion process shown in FIGS. Or an order change) to infer various implementation methods for converting the multi-focus two-dimensional image into a three-dimensional stereoscopic image, but the present invention includes all the inferred implementation methods. The technical features are not limited only to the implementation method shown in Fig. 9B.

9A and 9B, the focus information determiner 160 included in the 3D stereoscopic image converting apparatus using the multifocal 2D image illustrated in FIG. 1 is photographed by the image capturing unit 150. N (N> 1) objects included in the input preview image data are automatically recognized through pattern recognition (or image recognition) or included in the preview image data in association with the key input unit 175. By manually selecting N (N> 1) objects, the preview image data includes a boundary that is separated from the background (or other objects, such as humans, animals, or objects) in the preview image data. To determine N (N> 1) focal positions and the number of focal points for N (N> 1) objects (or focal regions) as shown in FIG. 2A or 2B. (900).

According to an exemplary embodiment of the present invention, when the identifiable object is not included in the preview image data, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions. Preferably, the center positions of the N (N> 1) focal regions are determined as N (N> 1) focal positions, and the number of N (N> 1) focal positions is N (N> 1) focal points. It is preferable to determine the number.

Here, the N (N> 1) pieces of focal number information is the maximum number of shots that the image capturing unit 150 can capture for a unit time, or the focus control unit 155 of the image capturing unit 150 for a unit time. At least one maximum control number for controlling the focus, or at least one maximum number of image data acquisition that the image acquisition unit 165 can obtain two-dimensional image data from the image capture unit 150 for a unit time It is preferable to set smaller than the limit focal number.

According to the exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the vertical direction as shown in FIGS. 9A and 9BA. As described above, it is preferable to determine the center positions of the N (N> 1) focal regions divided in the vertical direction as the N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions in the left and right directions, and the N divided in the left and right directions as described above. It is preferable to determine the center positions of the (N> 1) focal regions as N (N> 1) focal positions.

According to another exemplary embodiment of the present invention, the focus information determiner 160 divides the preview image data into N (N> 1) focal regions according to a grid shape, and divides the preview image data into a grid shape as described above. It is preferable to determine the center position of the N (N> 1) focusing areas as N (N> 1) focusing positions.

According to another exemplary embodiment of the present invention, when the identifiable object is included in the preview image data, the focus information determiner 160 may include the centers of N (N> 1) objects included in the preview image data. Preferably, the position is determined to be N (N> 1) focal positions, and the number of N (N> 1) focal positions is preferably determined to be N (N> 1) focal positions.

Here, the N (N> 1) pieces of focal number information is the maximum number of shots that the image capturing unit 150 can take for a unit time (for example, 50 ms, etc.) or the focus control unit 155 takes the image for a unit time. Maximum number of control times for controlling the focus of the photographing unit 150 or maximum number of image data acquisition units for obtaining the 2D image data from the image photographing unit 150 by the image obtaining unit 165 for a unit time. It is preferably set smaller than the limit focal number including at least one.

According to an exemplary embodiment of the present disclosure, when the number of focus positions (or the number of focal points) included in the preview image data is greater than the limit focal number, the focus information photographing unit may include the preview image among the objects included in the preview image data. It is preferable to determine N (N> 1) focal positions for objects within the above limit focal number among objects at the center position (or top, bottom, left, or right) of the viewing image data.

According to another exemplary embodiment of the present invention, when the number of focus positions (or the number of focus points) included in the preview image data is less than the limit focus number, the focus information photographing unit includes N (N) included in the preview image data. It is desirable to determine N (N> 1) focal positions for> 1) objects.

When the N (N> 1) focal position information and the N (N> 1) focal number are determined by the focus information determiner 160, the focus controller 155 may focus on the image capturing unit 150. Control to autofocus on a first focus position among the N (N> 1) focus positions to capture a first 2D image through the image capturing unit 150 (905), and the image obtaining unit 165 ) Acquires the first two-dimensional image data for the first focusing position (910), and correspondingly, the focus controller 155 confirms the first focus control information for the first auto focus (915). In the process, until N (N> 1) two-dimensional image data for N (N> 1) focal positions are obtained, and N (N> 1) focal position focus control information is confirmed (925), It is repeatedly performed 920 for the nth (n = 2, 3, ..., N) focal position.

When the N (N> 1) two-dimensional image data is obtained through the process, and the N (N> 1) focal position focus control information is confirmed (925), the focal length calculator 110 determines the identified N As shown in FIG. 3A or FIG. 3B using (N> 1) pieces of focus control information (eg, lens separation distance information for auto focus, refractive index information (or magnification information) of the spaced lens, etc.) The focal length between the object (or focal region) included in the N (N> 1) two-dimensional image data and the lens is calculated (930).

If the focal lengths for the N (N> 1) two-dimensional image data are calculated (935), the focus overlapping processing unit 115 provides the focal position information corresponding to the N (N> 1) two-dimensional image data. In operation 940, the object (or the focus area) corresponding to the focus position is aligned in line of sight and overlapped as shown in FIG. 4A or 4B.

If the N (N> 1) two-dimensional image data are superimposed (945), the depth calculator 120 may be arranged in the order in which the focal lengths for the N (N> 1) two-dimensional image data are close to each other. Arranged to calculate depth values for N (N> 1) two-dimensional image data (950), wherein one two-dimensional image data (or an object corresponding to a focal position of the two-dimensional image data (or focus area)) Preferably, one depth value is set at.

According to an exemplary embodiment of the present invention, a depth value of two-dimensional image data having close focal lengths for N (N> 1) two-dimensional image data is set to '1', and the two-dimensional closest focal length is two-dimensional. Preferably, the depth value of the image data is set to '2', and the depth value of the two-dimensional image data having the furthest focal length is set to 'N'.

If depth values of the N (N> 1) two-dimensional image data are calculated (955), the parallax information checking unit determines the disparity angle information corresponding to the left and right eyes according to a distance recognition method through the left and right eyes of a human being. Check (960).

According to the exemplary embodiment of the present invention, the parallax information checking unit has N (N> 1) focal lengths for N (N> 1) two-dimensional image data according to the depth value calculated by the focal length calculating unit 110. And N (N> 1) two-dimensional image data (or N (N) based on the binocular distance between the left eye and the right eye (for example, the average binocular distance according to the average face size, which can be set through the key input unit 175). It is preferable to calculate (or confirm) N (N> 1) parallax angle information for an object (or a focus area) corresponding to a focus position of> 1) two-dimensional image data.

If the disparity angle information corresponding to the left eye and the right eye is confirmed (965), the reduction ratio calculating unit 130 triangulates the disparity angle information and the focal length information corresponding to the left eye image data and the right eye image data. In operation 970, the reduction ratio for reducing and correcting the image data distorted in the horizontal direction is calculated by substituting the logarithmic function.

According to the exemplary embodiment of the present invention, the reduction ratio calculator 130 disparity angle information corresponding to the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focus position of the two-dimensional image data). And a focal length information is substituted into a trigonometric function (or logarithmic function) to calculate a reduction ratio for the left eye image data and the right eye image data.

According to another exemplary embodiment of the present invention, the reduction ratio calculator 130 may include N (N> 1) two-dimensional image data (or two-dimensional image data) used to generate the left eye image data and the right eye image data. The parallax angle information and the focal length information corresponding to any two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) of the object corresponding to the focal position It is possible to calculate a reduction ratio for the left eye image data and the right eye image data by substituting a trigonometric function (or logarithmic function), and the present invention is not limited thereto.

If the reduction ratio is calculated (975), the input image reduction unit 105 checks the transform size (or resolution) corresponding to the number N of 2D image data, and inputs N (N> 1). Comparing the size (or resolution) of the 2D image data with the checked size (or resolution) to determine whether to reduce or reduce the size (or resolution) of the N (N> 1) 2D image data (980). ).

According to the exemplary embodiment of the present invention, the input image reduction unit 105 may convert the left / right image data within a preset conversion time (or corresponding to the data processing speed of the image conversion module 100). Resolution).

Here, if the data processing speed of the image conversion module 100 is constant, the shorter the conversion time, the smaller the transform size (or resolution), and the longer the conversion time, the larger the transform size (or resolution), The larger the number N of 2D image data, the smaller the transform size (or resolution), and the smaller the number N of the 2D image data, the larger the transform size (or resolution).

For example, if the N (N> 1) two-dimensional image data input through the image acquisition unit 165 is one still image, the conversion time may be longer than several seconds, but the image acquisition unit 165 may be If the N (N> 1) two-dimensional image data input through the video, the conversion time should be short within a few to several tens of milliseconds corresponding to the number of frames per second.

When the conversion size (or resolution) is confirmed, the input image reduction unit 105 confirms the size (or resolution) of N (N> 1) two-dimensional image data input from the image acquisition unit 165. The size (or resolution) of the N (N> 1) two-dimensional image data input from the image acquisition unit 165 is determined as the transformed size (or Resolution), the input image reduction unit 105 may not reduce the N (N> 1) two-dimensional image data.

If the result of the check does not reduce or reduce the size (or resolution) of the N (N> 1) two-dimensional image data (982), the input image reduction unit 105 is the obtained N (N> 1) The size (or resolution) of the 2D image data is reduced and converted according to the calculated reduction ratio (984).

On the other hand, if it is determined that the size (or resolution) of the N (N> 1) two-dimensional image data is reduced and converted (982), the input image reduction unit 105 is the obtained N (N> 1) The size (or resolution) of the 2D image data is reduced and converted according to the conversion size (or resolution), and then reduced and converted according to the calculated reduction ratio (986).

The depth image synthesizing unit 125 arranges N (N> 1) two-dimensional image data of which the focal points are overlapped according to the depth value, and arranges the N (N> 1) two-dimensional image data arranged. In operation 988, short-circuit two-dimensional image data is generated by combining the depth value.

According to the exemplary embodiment of the present invention, the depth value image synthesis unit 125 may include a 2D image corresponding to an mth (1 <= m <= N) depth value among N (N> 1) two-dimensional image data. Two-dimensional image data corresponding to the mth (0 <= m <N) object (or focus area) corresponding to the focal position of the data sharply and corresponding to the mth (1 <= m <= N) depth value Process another object (or focus area) in the transparent (or mth (1 <= m <= N) direction as the distance from the object (or focus area) increases), and the m (1 <= m <= N) In addition to the depth value, k (0 <= k <N, m! = K) corresponds to the focus position of the two-dimensional image data corresponding to the depth value k (0 <= k <N, m! = k) Clearly processes an object (or a focus area) and transparently clears another object (or focus area) area in the 2D image data corresponding to the k-th (0 <= k <N, m! = k) depth value. (Or transparency increases as you move away from the k (0 <= k <N, m! = K) object (or focus area)) The 2D image data corresponding to the mth (1 <= m <= N) depth value and the 2D image data corresponding to the kth (0 <= k <N, m! = K) depth value By combining, it is preferable to synthesize so that N (N> 1) objects (or focus areas) included in the N (N> 1) two-dimensional image data are clearly output.

The parallax information checking unit checks the parallax processing information corresponding to the left eye and the right eye with respect to the shorted 2D image data through the parallax angle information, the focal length, and the depth value (990).

Also, as shown in FIG. 7, the parallax information checking unit (or focus area) corresponds to a focal position of N (N> 1) two-dimensional image data (or N (N> 1) two-dimensional image data). It is preferable to identify the zero-parallel target two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) to be processed by zero parallax.

According to an exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focus area) corresponding to a focal position of the 2D image data) includes N (N> 1) pieces of 2D image data ( Or among the N (N> 1) pieces of two-dimensional image data (or an object corresponding to the focal point), the focal length is located in the middle (for example, the middle of the closest and farthest focal lengths) Or two-dimensional image data (or an object corresponding to a focal position of the two-dimensional image data) whose depth value is located in the middle (eg, the middle of the smallest depth value and the largest depth value). It is preferable to confirm with the said zero parallax object.

According to another exemplary embodiment of the present invention, the zero parallax target two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) includes N (N> 1) two-dimensional image data. (Or two-dimensional image data (or two-dimensional image) having the shortest focal length or the smallest depth value (or an object corresponding to the focal position of N (N> 1) two-dimensional image data). Preferably, the object (or focus area) corresponding to the focus position of the data is identified as the zero parallax object.

According to another exemplary embodiment of the present invention, the zero-parallel target 2D image data (or an object (or a focal region) corresponding to a focal position of the 2D image data) includes N (N> 1) 2D images. Two-dimensional image data (or two-dimensional) having the longest focal length or the largest depth value among the data (or objects (or focal regions) corresponding to the focal positions of N (N> 1) two-dimensional image data) Preferably, the object (or focus area) corresponding to the focus position of the image data is identified as the zero parallax object.

Also, as shown in FIG. 7, the disparity information checking unit has a focal length that is closer than the zero parallax target two-dimensional image data (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data), or Two-dimensional image data having a small depth value (or an object (or a focal region) corresponding to the focal position of the two-dimensional image data) corresponds to the two-dimensional image data (or the focal position of the two-dimensional image data) subject to positive parallex Object (or focus area), and the focal length is farther than the zero parallax target 2D image data (or the object (or focus area) corresponding to the focus position of the 2D image data), or the depth value is Large two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) is subjected to a negative parallex target two-dimensional image data (or second of two-dimensional image data). It is preferable to check with an object (or a focus area) corresponding to the point position.

Also, as shown in FIG. 7, the parallax information checking unit (or focus area) corresponds to a focal position of N (N> 1) two-dimensional image data (or N (N> 1) two-dimensional image data). N (N> 1) disparity angle information and focal length information for)) are substituted into a trigonometric function (or logarithmic function) to correspond to the focal position of the two-dimensional parallax object 2D image data (or 2D image data). Bilateral parallax pixel movement information for an object (or focal region) and parallax pixel for the parallax target 2D image data (or an object (or focal region) corresponding to a focal position of the 2D image data) It is desirable to confirm the movement information.

According to the exemplary embodiment of the present invention, the parallax processing information checked by the parallax information confirming unit may correspond to the zero parallax target 2D image data (or the focal position of the 2D image data included in the 2D image data). Object (or focal region)), at least one or more parallax target two-dimensional image data (or an object (or focal region) corresponding to a focal position of the two-dimensional image data), bi-parallel direction pixel movement information, and at least one or more Preferably, the parallax object includes two-dimensional image data (or an object (or a focus area) corresponding to a focal position of the two-dimensional image data) and the parallax direction pixel movement information.

If the parallax processing information is confirmed (992), the left and right image generating unit 140 as shown in Figure 7 the two-dimensional image data (or two-dimensional) of the two parallax object included in the synthesized two-dimensional image data A pixel position corresponding to an object (or a focal region) corresponding to a focal position of the image data is moved in the left and right directions according to the parallax direction pixel movement information, and is further included in the synthesized two-dimensional image data. By moving the pixel position corresponding to the parallax target two-dimensional image data (or an object (or a focus area) corresponding to the focal position of the two-dimensional image data) in the left and right directions according to the disparity direction pixel movement information, The left eye image data and the right eye image data of the synthesized 2D image data are generated (994).

If left eye image data and right eye image data for the synthesized 2D image data are generated (996), the image output unit 190 outputs the generated left eye image data and right eye image data to the screen output unit 180. Or through the communication unit 195, or the image storage unit 193 matches the generated left eye image data and right eye image data in one-to-one correspondence and stores them in the storage medium 183 (998). ).

According to the present invention, a digital camera (or video camera) equipped with an auto focus function is configured to capture a multi-focus two-dimensional image at high speed according to a focal position and the number of focal points, and the photographed plurality of multi-focus two-dimensional images are taken. By having a function of converting to a 3D stereoscopic image by synthesizing according to the focal length and the depth value, there is an advantage to produce a cheap and natural 3D stereoscopic image.

Claims (10)

Determining N (N> 1) focal positions and the number of foci of the two-dimensional image; When the N (N> 1) multi-focus two-dimensional image data corresponding to the determined N (N> 1) focal positions and the number of foci are acquired, the N (N> 1) multi-focus two-dimensional image data is obtained. Calculating N (N> 1) focal lengths; Calculating N (N> 1) depth values in inverse proportion to the calculated N (N> 1) focal lengths; Synthesizing N (N> 1) multifocal two-dimensional image data according to N (N> 1) depth values; The zero parallax image, the positive parallax image data, and the negative parallax image data are identified among the N (N> 1) multi-focus 2D image data. Confirms bi-parallel direction pixel movement information based on a disparity angle with respect to at least one disparity image data based on a difference, and based on the disparity angle with respect to at least one disparity image data based on the identified zero disparity Confirming disparity direction pixel movement information; And The left eye image data and the right eye image data are generated by applying the identified bilateral parallax direction pixel movement information to at least one bilateral parallax image data, and applying the identified parallax direction pixel movement information to at least one parallax image data. 3D stereoscopic image conversion method using a multi-focus two-dimensional image, characterized in that it comprises a. The method of claim 1, When the N (N> 1) multi-focus 2D image data corresponding to the determined N (N> 1) focal positions and the number of foci are acquired, the size (or And converting the resolution into a preset conversion size (or resolution). The method of claim 1, And superimposing so that the focal positions of the N (N> 1) two-dimensional image data are aligned along the line of sight. The method of claim 1, And calculating a reduction ratio for correcting horizontal distortion of the left and right image data based on the parallax angle information and the focal length information corresponding to the left eye image data and the right eye image data. 3D stereoscopic image conversion method using 2D image. A recording medium comprising a program for executing the method of claim 1. A focal number determiner for determining N (N> 1) focal positions and the number of foci of the two-dimensional image; A focus control unit controlling multiple focuses for N (N> 1) focal positions by N (N> 1) focal points; An image obtaining unit obtaining N (N> 1) multi-focus two-dimensional image data by the focus control; A focal length calculator configured to calculate N (N> 1) focal lengths for N (N> 1) multifocal two-dimensional image data; A depth value calculator for calculating N (N> 1) depth values in inverse proportion to the calculated N (N> 1) focal lengths; A depth image synthesizing unit for synthesizing N (N> 1) multifocal two-dimensional image data according to N (N> 1) depth values; Of the N (N> 1) multi-focus two-dimensional image data, zero parallax image data, positive parallax image data and negative parallax image data are checked, and the identified zero parallax Determine the bi-parallel direction pixel movement information based on the parallax angles for the at least one positive parallax image data, and based on the parallax angles for the at least one negative parallax image data based on the identified zero parallax. A parallax information checking unit for checking parallax direction pixel movement information; And The left eye image data and the right eye image data are generated by applying the identified bilateral parallax direction pixel movement information to at least one bilateral parallax image data, and applying the identified parallax direction pixel movement information to at least one parallax image data. And a left and right image generating unit. The method of claim 6, 3, further comprising an input image reduction unit for converting the size (or resolution) of the N (N> 1) two-dimensional image data into a preset conversion size (or resolution). 3D stereoscopic image conversion method. The method of claim 6, And a focus overlapping processor overlapping the focus positions of the N (N> 1) two-dimensional image data so as to be aligned along the line of sight. The method of claim 6, And a reduction ratio calculator for calculating a reduction ratio for correcting horizontal distortion of the left and right image data based on the parallax angle information and the focal length information corresponding to the left eye image data and the right eye image data. 3D stereoscopic image conversion device using multi-focus 2D image. The method of claim 6, wherein the focus number determiner, Through pattern recognition (or image recognition) on N (N> 1) objects included in 2D image data, the background (such as N (N> 1) objects-people, animals, objects, etc. Or subjects with boundaries that separate them from other objects), When N (N> 1) objects are confirmed as the result of the check, the center position of the identified object is determined as N (N> 1) focal positions, If the object is not confirmed as a result of the checking, the 2D image data is divided into N (N> 1) focal regions in the vertical direction (or the horizontal direction or the lattice direction), and the center position of the divided focal regions is determined. 3D stereoscopic image conversion device using a multi-focus two-dimensional image, characterized in that determined by the N (N> 1) focal positions.

Images