WO2013077508A1 - Device and method for depth map generation and device and method using same for 3d image conversion - Google Patents

Device and method for depth map generation and device and method using same for 3d image conversion Download PDF

Info

Publication number
WO2013077508A1
WO2013077508A1 PCT/KR2012/003347 KR2012003347W WO2013077508A1 WO 2013077508 A1 WO2013077508 A1 WO 2013077508A1 KR 2012003347 W KR2012003347 W KR 2012003347W WO 2013077508 A1 WO2013077508 A1 WO 2013077508A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
depth map
image
value
initial
input image
Prior art date
Application number
PCT/KR2012/003347
Other languages
French (fr)
Korean (ko)
Inventor
우대식
김종대
박재범
전병기
정원석
Original Assignee
에스케이플래닛 주식회사
시모스 미디어텍(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/128Adjusting depth or disparity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Abstract

The present invention relates to a device and method for depth map generation and to a device and method using same for 3D image conversion. The depth map generation device includes: a feature information extraction part extracting at least one piece of feature information with respect to an input image; a depth map initialization part generating a depth map with respect to the input image on the basis of the feature information; an FFT conversion part performing an FFT with respect to the conversion of the input image into a frequency image; and a depth map determination part evaluating a correlation value by using a representative value of the frequency image and the mean value of the initial depth map to determine a final depth map on the basis of the correlation value. According to the present invention, the error of the depth map for expressing a 3D image generated when an image is automatically converted may be corrected.

Description

Three-dimensional image conversion device and method using the same and the depth map generation apparatus and method

The present invention is a depth map generation related to a three-dimensional image conversion device and method using the same and an apparatus and method, and more particularly, to extract at least one characteristic information about the input image, based on the characteristic information of the input image to generate the initial depth map, after performing the FFT (Fast Fourier transform) to transform into the frequency image for the input image, by using the average value of the representative value and the initial depth map of the frequency image to obtain a correlation value, on the basis of the correlation value it relates to a three-dimensional image conversion device and method using the same and the depth map generation apparatus and method to determine the final depth map.

As the recently amplified interest in the 3D image (Stereoscopic image), it has been actively studied for the 3D image.

In general, humans are known to feel the cubic effect by the time difference between both eyes most. Accordingly, 3D video may be implemented using a human such properties. For example, a particular subject, a left-eye image and the viewer's to distinguish the right-eye image viewed through the right eye, the viewer has the particular subject by displaying the left-eye image and the right-eye image at the same time as shown by the left eye of the viewer with a 3D image you can so that you can see. After all, 3D images can be implemented by displaying it by making the both eyes (binocular) image divided into a left-eye image and right-eye images.

In order to convert depth information without a monocular (monocular) 2D image into a 3D image it is required the task of rendering (rendering) in addition to the depth information for the 2D image.

Typically, the three-dimensional transformation is divided into manual mode and automatic mode. Passive way is to create a depth map So while watching the video on the subjective judgment of a person for literally any video. This process is watching the footage is based on the subjective judgment of people you can expect the depth map, even small parts of the video. Therefore, it is the direct production of a depth map for each person in the footage, errors of fact, the depth map is very small. However, a lot of time and effort is required to create a depth map because of the footage directly to the people involved in every video.

Autostereoscopic conversion means for analyzing characteristics of the image to extract the proper depth map, and generate the left and right stereoscopic image by using this. In this process, the video object itself since there is no information on the depth map utilizing conventional image features such as the outer (Edge) characteristics of the image, color, brightness characteristics, the vanishing point characteristics and generates the depth map. However, these features are not the effect of three-dimensional large because in many cases does not match the characteristics of the three-dimensional image with the footage itself.

And, inside a video object it will include a video content in various forms and comprises a depth map, too many errors obtained Extracting depth maps through an image processing on the contents of each video object, and practically impossible, through the image processing.

Error of the depth map obtained through such image processing can be classified into two types.

One of the error or reversal of the depth map of the partial area of ​​the video object, and a combination thereof and the other is an error or a reversal of the entire depth map video. Of course, errors of this depth map is not quite so simple to distinguish the technical methods such as image processing.

Thus, there is a need for a technology that can detect an error in the depth map automatically from an objective point of view has only video.

The present invention been made in view of solving the above problems, an object of the present invention is a depth capable of correcting the error in the depth map (Depth Map) for a three-dimensional expression generated from the auto-stereoscopic conversion process in the video object map generation apparatus and to provide a method and a stereoscopic image using the same transformation method and apparatus.

Another object of the present invention provides a three-dimensional image conversion device and method using the same errors in the depth map of the image generated by the image processing for automatic three-dimensional image transformation and objective detection and correction depth map generation apparatus and method for a have.

And It is another object of the present invention corrects the error of the depth map, and by using the corrected depth map convert the 2-D image into a 3-D image and to minimize the error in the image conversion depth map generation apparatus and method and it is to provide a three-dimensional image conversion apparatus and method.

According to an aspect of the invention, characteristic to extract at least one characteristic information on the input image information extracting unit, unit based on the attribute information initializing the depth map to generate an initial depth map for the input image, the input image performing a FFT (Fast Fourier transform) for by using the average value of the representative value and the initial depth map for the FFT converter, the frequency image to be converted to a frequency image to obtain a correlation value, the end on the basis of the correlation values the depth map generation apparatus comprising: a decision depth map for determining a depth map is provided.

* Is the characteristic data extraction unit extracting characteristic information including the boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, at least one of a histogram (histogram) information.

The depth map initialization unit at least after separated by one block (block) the initial depth by setting the initial depth values ​​(initial depth) for the at least one block of a map of pixels (pixel) a plurality of constituting the input image ( It generates a depth map).

After the depth map determination section calculated the average of the depth values ​​corresponding to a block area that matches the area obtained for the representative value to obtain a representative value of the combined pixel values ​​corresponding to the high frequency area in the frequency image, in the initial depth map, the representative by using the value and the average value is obtained for the correlation values.

The depth map determination section using the equation to calculate the correlation value (CRV (Co-Relation Value)).

Formula;

CRV = Σ (FFT (n) * Depth (n))

Here, the FFT (n) is a representative value representing the block definition of the frequency image, and the Depth (n) is the average value of the initial depth map corresponding to a block area that matches the block area of ​​the FFT (n), n is means an index for each block (block).

Further, when the depth map determination section wherein the correlation value is not less than a predetermined threshold, and determining the initial depth map to a final depth map, other than the threshold value or more by inverting the depth value of the initial depth map that the inverted depth to determine the final depth map to map.

According to another aspect of the invention, and to generate the initial depth map for the input image based on the image analysis unit, the characteristic data to extract at least one characteristic information by analyzing the input image of the two-dimensional, in the input image after conversion by performing the FFT to the frequency image for setting the depth map by using the average value of the representative value and the initial depth map of the frequency image to obtain a correlation value, and determine the final depth map on the basis of the correlation values the three-dimensional video conversion device including unit, by using the end parts of the three-dimensional depth map image generator for converting the input image into a three-dimensional image of the three-dimensional is provided.

The depth map setting unit of the FFT converter, the frequency image conversion on the basis of the characteristic data to the frequency image by performing FFT on the depth map initialization unit, the input image to generate the initial depth map for the input image using the average value of the representative value and the initial depth map to obtain a correlation value, and comprising a decision depth map to determine the final depth map on the basis of the correlation value.

In accordance with another aspect of the invention, the depth map generation unit depth according to a method of generating a map comprising the steps of: extracting at least one feature information about the input image, the initial depth for the input image based on the attribute information generating a map, the method to obtain the step of performing an FFT to transform the frequency image for the input image, a correlation with a mean value of the representative value and the initial depth map of the frequency image relationship values, the correlation values the depth map generation method determining a final depth map is provided as a basis.

Obtaining a correlation value by using the average value of the representative value and the initial depth map of the frequency image, the step of obtaining the representative value of the combined pixel values ​​corresponding to the high frequency area in the frequency image, the representative at the initial depth map and finding an average of the depth values ​​corresponding to a block area that matches the value obtained with the area, and a step of obtaining a correlation value using the calculated representative value and the average value.

Determining a final depth map on the basis of the correlation value, if the correlation value is not less than a predetermined threshold, and determining the initial depth map to a final depth map, other than the threshold value or more of the initial depth map depth reversal values ​​by characterizes in determining the depth map by inverting the final depth map.

In accordance with another aspect of the invention, performing FFT on the phase, the input image to generate the initial depth map for the input image on the basis of the step, the characteristic information for extracting the at least one property information for the input image the depth map comprising the step of determining the final depth map by obtaining a correlation value by using the average value of the representative value and the initial depth map of the frequency image, based on the correlation value to be converted to a frequency image generating method is recorded to the program is provided with a recording medium readable in the electronic device.

In accordance with another aspect of the invention, there is provided a method of three-dimensional image converting unit converts the autostereoscopic image, the analyzes an input image of an inputted two-dimensional basis of the steps, the characteristic data to extract at least one characteristic information to generate the initial depth map for the input image, and using the step, the final depth map to determine the final depth map by checking the validity whether the initial depth map for converting the input image into a three-dimensional image of a three-dimensional the three-dimensional image conversion method comprises the step is provided.

Determining the final depth map comprises the steps of converting the frequency image by performing the FFT on the phase, the input image to generate the initial depth map for the input image based on the characteristic information, a high frequency in the frequency picture obtaining a representative value of the combined pixel values ​​corresponding to the region, and the step from the initial depth map to obtain the average of the depth values ​​corresponding to a block area that matches the area obtained for the representative value, representative of the frequency image value and the initial obtaining a correlation value by using the average value of the depth map, the correlation value is predetermined not less than the threshold value, the depth value of the initial depth map if the initial depth map, not determined by the final depth map, and the threshold value or more reversal of determining the inverted depth map to a final depth map.

In accordance with another aspect of the invention, by analyzing an input image of the input 2-D and generate an initial depth map for the input image on the basis of the phase, the characteristics of extracting at least one feature information, the initial depth determining a final depth map by checking the validity whether the map, the three-dimensional image conversion method comprising the step of using the final depth map converting the input image into a three-dimensional image of the three-dimensional is recorded as a program in an electronic device the readable storage medium is provided.

Therefore, according to the present invention, it is possible to correct errors of the depth map (Depth Map) for a three-dimensional expression generated from the auto-stereoscopic conversion process in the video object.

Further, it is possible to objectively detect and correct the error in the depth map of the image generated through the auto-stereoscopic image conversion processing.

In addition, correct errors in the depth map, and it is possible to minimize the error of the image conversion to convert the 2-D image into a 3-D image using the corrected depth map.

1 is a block diagram showing the configuration of a three-dimensional video conversion device according to the present invention.

2 is a block diagram schematically showing the configuration of a depth map generation unit according to the invention.

3 is a case to convert the input image according to the present invention as the stereoscopic images of three-dimensional an illustration for explaining the difference between the initial depth map and the final depth map.

Figure 4 is an exemplary view illustrating an image of the FFT frequency component according to the invention.

Figure 5 is illustrative of a block diagram representation of a 8x8 block according to the present invention into a frequency domain through a FFT.

Figure 6 is a view showing how the three-dimensional image conversion apparatus according to the invention converts the input image of the two-dimensional three-dimensional image of a three-dimensional.

7 is a flowchart illustrating a method of generating a depth map generation unit depth map in accordance with the present invention.

With reference to the accompanying drawings, the present will be described in more detail a preferred embodiment of the invention. Description of the same components in the following description with reference to the accompanying drawings or corresponding are assigned the same reference numerals and a duplicate thereof will be omitted.

Hereinafter, to generate a depth map representing the three-dimensional and corrected for the video object that satisfies the two prerequisites below.

1 is a prerequisite "It is typical in light color or a strong brightness is often a close distance." For example, if a lot more acid when compared to away mountains that close while viewing the picture seems to be converted into an achromatic color. Or the bright color of the strong object more far away quality when it is about saturated colors. These features are characteristic of human vision, but also is also a feature of a camera for capturing them.

Prerequisite 2 is "typically the case with many bright parts in the picture that close." For example, when the far and when the same objects are near a sharpness of an object varies This occurs due to the limitation of the resolution of the camera for visually or recording.

The present invention is directed to the converter and correcting it by using the three-dimensional two properties as described above. 1 is a prerequisite condition for converting the three-dimensional, two prerequisites is a means for correcting it. That is, it extracts a depth map with the color, brightness information of the video object, and detect an error in the depth map with the objective verification of such a depth map in part or entirely on whether a valid and correct them.

1 is a block diagram showing the configuration of a three-dimensional video conversion device according to the present invention.

1, a three-dimensional image conversion device 100 comprises an image analysis unit 110, a depth map setting unit 120, a three-dimensional image generation unit 130.

The image analysis unit 110 analyzes the input image of the two-dimensional and extracts at least one feature information. The characteristic information includes a boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, a histogram (histogram) information, and the like.

And the depth map setting unit 120 generates an initial depth map for the input image based on the attribute information extracted by the image analysis unit 110, and performs FFT (Fast Fourier Transform) on the input image after converting to a frequency image by using the average value of the representative value and the initial depth map for the image frequency to obtain a correlation value, and determines the final depth map on the basis of the correlation value.

A detailed description of the depth map setting unit 120 will be described with reference to Fig.

The three-dimensional image generation section 130 is converted by the final depth map is determined by the setting unit 120, the depth map for the input image of the two-dimensional to three-dimensional image of a three-dimensional. For example, the three-dimensional image generation section 130 may generate the time difference (parallax) information using the final depth map, and generating a 3D stereoscopic image based on the time difference information. The generated three-dimensional image appears to allow more depth (depth) value for each of the pixels to vary in each frame, more three-dimensional appearance.

Here, the input to the three-dimensional image generation unit 130 has been described as using the time difference information converting a two-dimensional image into a three-dimensional image of a three-dimensional, the three-dimensional image generation unit 130 is used for the final depth map image how to convert the three-dimensional image are different according to prior methods.

Three-dimensional image conversion device 100 configured as described above can be converted, by setting the depth values ​​for the input image based on the characteristic information of the input image, the input 2D image of the 3D image (Stereoscopic Video).

Figure 2 explains the difference between the initial depth map and the final depth map when converting the input image according to the third is the present invention, a block diagram schematically showing the configuration of a depth map generation unit according to the invention with three-dimensional image of a three-dimensional for example, Fig. 4 is an exemplary view, Figure 5 is illustrative of a block diagram representation of a 8x8 block according to the present invention into a frequency domain through the FFT illustrating an image of the FFT frequency component according to the invention.

In Figure 1 has been described to depth map setting part, be described with the depth map generating unit 200 in Fig.

2, the depth map generation apparatus 200 includes a characteristic data extraction unit 210, the depth map initialization unit (220), (Fast Fourier Transform), FFT conversion unit 230, determines the depth map 240 It includes.

The characteristic data extraction unit 210 extracts at least one property information for the input image. Here, the input image can be a monocular (monocular) image.

Attribute that is the characteristic data extraction unit 210 extracts the information may be a boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, or histogram (histogram) information, and the like.

The characteristic data extraction unit 210 extracts a pixel (pixel) or the block (block) in the image attribute information through a variety of analysis methods of the unit to collect information that is the basis for the depth map generation.

The depth map initialization unit 220 generates an initial depth map for the input image based on the attribute information extracted by the characteristic data extraction unit 210.

That is, the depth map initialization unit 220 has at least one block (block) by one minutes after the initial depth value for the at least one block (initial depth), a plurality of pixels (pixel) forming the input image set to generate the initial depth map (depth map).

The depth map initialization unit 220 generates a depth map (depth map) for each frame of the two-dimensional image based on the extracted characteristic information. That is, the depth map initialization unit 220 is to extract the depth value for each pixel for each frame from depth maps (depth map) for the 2-D image. Here, the depth map is a data structure storing a depth value for each pixel per frame for the two-dimensional image (depth value).

For example by converting the input image into a three-dimensional image of a three-dimensional using the initial depth map generated by the depth map initialization unit 220 Referring to Figure 3 will be described.

Relative to the observer spider closest situated in the observer, then there is located a flower, the background will be described as an example in the farthest image from the observer, the image can be a spider, flowers, background separation in three steps according to a position Do. Of course more details from a possible, but will be divided into three stages as an example for description.

It is possible in this manner to the visual characteristics of subjective human analysis, but, if it be analyzed automatically through image processing using the prerequisite one spider that is considered to be the front of the achromatic and the depth map that is farthest because the darkness It will be represented.

Figure 3a is a view showing a three-dimensional image transformation to the depth map in accordance with the preconditions 1.

Further, it jundamyeon change the depth map generated by the depth map initialization unit 220, such that the film video object as reverse printing of the photo film can be formed of a normal depth map.

That is, the depth map of the image is the initial depth map of Figure 3a when reversed as print film can be converted to depth map as shown in FIG 3b, and Figure 3b is capable of less than three-dimensional representation error Figure 3a.

Figure 3b is an image of spider so the spider to the front and brightly expressed, followed by expression is dark flower region and background is due to the degree of change is substantially not. As a result, when selecting a depth map of Figure 3b, while improving the error can be expressed in a normal three-dimensional.

Accordingly, the depth map generation apparatus 200 may select the image in the depth map of Fig. 3a to select whether or depth map of Figure 3b to be converted to determine conversion.

The frequency image and the input image performs the FFT (Fast Fourier Transform) conversion, to determine whether the depth map generation unit 200 is converted by selecting the depth map of Fig. 3a selected by converting the depth map whether or Figure 3b of produced, and by using the image frequency will determine the final depth map.

Accordingly, the depth map generation apparatus 200 includes a FFT conversion unit 230, determines the depth map 240.

The FFT conversion unit 230 converts the input image by performing an FFT on the input image to the image frequency in the frequency domain. That is, the FFT conversion unit 230 converts the input image on the spatial frequency in the image.

It will be the FFT conversion unit 230, a reference to Figure 4 for one to perform a FFT converts the input image into a video image frequency. 4, the more light portion has a vivid or strong outside. That is, the high-frequency region, i.e., the sharpness or when expressing part representing a strong outer shell to the light color to be outside of the spider brightly represented in the image and then the flower region and background is expressed darker because of the degree of change is substantially not in the frequency domain do.

(A) of Fig image expression such as 4 is to the nature of the FFT, Fig. 5 (b) of Fig would image an arbitrary region of 8x8 pixel in the input image 5 is the image of (a) into a frequency domain through a FFT.

A region in (b) of Figure 5 is to say the low frequency band, region B refers to a high-frequency band. In other words, the region B is a high frequency component, when the image is sharper or has a strong outside becomes large, the value of the area B, the more simple the value of the upper region A larger value of the area B becomes relatively small.

The sum of the pixel values ​​belonging to the high frequency area in the frequency image, as, that of the total of the area B representing the sharpness in the 8x8 blocks (Block) expressing the variation between each pixel of the image large belonging to the partial area of ​​the image screen 8x8 It is defined as a representative value corresponding to the pixel. To a size value of each of the block units as the brightness value is the one image representation utilizing FFT high-frequency component as shown in Fig. 4 can be represented as images in this way.

The depth map determination section 240 obtains a correlation value by using the average value of the representative value of the frequency image generated by the FFT conversion unit 230 and the initial depth map, and the final depth map on the basis of the correlation values to be determined.

That is, the depth map determination section 240 calculates a representative value (FFT (n)) representing the sharpness of any block of 8x8 block units in the frequency image formed through the FFT conversion unit 230. Here, the representative value is obtained summing pixel values ​​corresponding to the high frequency area in the frequency image. Then determine the depth map 240, after obtaining the average value (Depth (n)) of the 8x8 block of the initial depth map matching block region of the FFT (n) by using the central value and the average value correlation calculate the value (CRV (Co-Relation value)).

The depth map determination section 240 by using the equation (1) calculate the correlation value (CRV (Co-Relation Value)).

Equation 1

CRV = Σ (FFT (n) * Depth (n))

Here, the FFT (n) is a representative value representing the block definition of the frequency image, and the Depth (n) is the average value of the initial depth map corresponding to a block area that matches the block area of ​​the FFT (n), n is It means an index for each block (block). The FFT (n) refers to the sum of the pixel values ​​corresponding to the high frequency region in the FFT-converted frequency image, the average value of the initial depth map from the initial depth map corresponding to a block area that matches the area obtained for the representative value, It means the average of the depth values.

The mathematical When the correlation value obtained by the equation 1, the depth map determination section 240 when the correlation value is not less than a predetermined threshold, and determining the initial depth map to a final depth map, other than the threshold value or more by inverting the depth value of the initial depth map determines that the inverted depth map to a final depth map.

As a result, by utilizing the frequency image conversion by the image in the frequency domain, such as the depth map determination section 240 is 4, the image automatically more appropriate for the depth map of Fig. 3b, rather than Fig. 3a to the depth map for converting the three-dimensional the It will draw that conclusion.

Also, if the depth map determining unit 240 matches the condition of the condition it is determined that in front of the more the color of the preconditions 1 strong and bright colors and sharp parts at the front, that is, when the correlation value is equal to or greater than a predetermined threshold, a prerequisite it is determined that the depth map generation of FIG. 3a as the first sense.

If it is determined that the correlation value in this case is not equal to or greater than the predetermined threshold value, a depth map The inversion phenomenon is reflected as the depth map determination section 240 Figure 3b more reasonable depth map.

Depth map generation unit 200 depth map extraction an initial depth map by using the attribute information such as color, brightness information of the video object, and through the objective verification of this initial depth map partial or whether entirely reasonable constructed as described above to detect errors and generate a final depth map.

6 is a view showing how the three-dimensional video conversion device according to the present invention converts an input image a two-dimensional three-dimensional image of a three-dimensional.

6, the three-dimensional image transformation apparatus extracts the at least one characteristic information by analyzing the input image of the two-dimensional (S602). Here, the attribute information includes a boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, a histogram (histogram) information, and the like.

After execution of the S602, the three-dimensional video conversion device generates an initial depth map for the input image based on the attribute information (S604).

Then the stereoscopic image converter determines the final depth map through the objective verification of said initial depth map whether partial or valid (S606).

That is, the three-dimensional image converting unit converts the frequency image by performing FFT on the input image. Then, the three-dimensional video conversion device calculates the representative value of the combined pixel values ​​belonging to the high frequency area in the frequency image, it calculates the average value of the depth values ​​corresponding to a block area that matches the area obtained for the representative value from the initial depth map. Then, the three-dimensional video conversion device calculates the correlation value using the equation (1) using the representative values ​​and the mean values ​​the obtained.

Then, the stereoscopic image converter determines the final depth map on the basis of the correlation value.

After execution of the S606 the three-dimensional image converting unit converts the input image using the final depth map with the determined three-dimensional image of a three-dimensional (S608).

7 is a flow chart illustrating a method of generating a depth map generation unit depth map in accordance with the present invention.

7, the depth map generation apparatus extracts the at least one property information for the input image, and generate the initial depth map for the input image based on the extracted characteristic information (S702). That is, the depth map generation apparatus extracts the characteristic information, such as the boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, a histogram (histogram) information. Then, the depth map generation apparatus may initially set the at least one block and then separated by a (block) the initial depth value for the at least one block (initial depth) of a plurality of pixels (pixel) forming the input image depth to generate a map (depth map).

After execution of the S702, the depth map generation apparatus may be transformed to the frequency image by performing FFT on the input image (S704).

After execution of the S704, the depth map generation unit may determine whether or not the image to the image frequency of the converted satisfy the precondition (S706). Here, the precondition is a two prerequisites, a prerequisite 2 "typically many, if the bright portion in the image is close to."

If the image that is determined in the S706 satisfy the precondition, the depth map generation apparatus may calculate the correlation value using the average value of the representative value and the initial depth map for the image frequency (S708). A detailed description of how to obtain the correlation value is illustrated in FIG.

After execution of the S708, the depth map generation unit may determine whether or not the calculated correlation value is greater than or equal to a predetermined threshold (S710).

If more than the threshold value determined result of the S710, the depth map generation apparatus may determine the initial depth map to a final depth map (S712).

If it is determined in the S710 the correlation value is not more than the threshold value, the depth map generation apparatus final depth map a depth map consisting of the inverting the depth value of the initial depth map and (S714), in the inverted depth values determines a (S716).

A computer program (also known as programs, software, software application, script, or code) for executing the method according to the present are included in the depth map generation apparatus or the three-dimensional video conversion device according to the present invention invention is a compiled or interpreted language or a priori or procedures, and can be written in any form of programming language, including the language, can also be deployed in any form, including other unit suitable for use in stand-alone program or a module, component, subroutine, or the computer environment. A computer program does not necessarily have to correspond to the files in the file system. The program part of the inside in a single file that is provided to the requesting program, or the multiple cross-file serving (e. G., One or more modules, sub programs, or file for storing part of the code), or a file that holds other programs or data may be stored in the (e. g., one or more scripts stored in a markup language document). A computer program can be deployed so as to be located at one site or distributed across multiple sites running on a computer of a cross-multiple-access computer or one by a communication network.

Medium readable by a suitable computer, for storing computer program instructions and data, for example, EPROM, EEPROM, and flash semiconductor memory device, such as a memory device, such as magnetic disks, such as internal hard disks and external disks, magneto-optical disks and CD-ROM and it shall include the DVD-ROM disc, including all forms of non volatile memory, media and memory devices. The processor and the memory is supplemented by a logic circuit or a special-purpose, can be integrated into it.

Implementation of the functional operation and the topics discussed in this specification be implemented in digital electronic circuitry, computer software, including the structure and structural equivalents disclosed herein, implemented in firmware or hardware, or implemented with one or more combinations of these It can be. Implementation may be implemented within one or more computer program products of the subject described herein, that is to say one or more modules of the computer program instructions encoded on the type of program storage medium to or for the execution of this controlling the operation of a data processing device as it may be implemented.

Implementation of the subject matter described herein includes, for example, a back-end component, such as a data server, or, for example, includes a middleware component, such as an application server, or, for example, your Web browser can interact implementation and mutual subject matter described in this specification or the graphical user It may be implemented in front-end component, or such back-end, computing system comprising at least one combination of all of the middleware or a front-end component, such as a client computer having an interface. Of system components, for example it can be interconnected by any form or medium of digital data communication such as a communication network.

The specification includes a number of specific implementations of the detail, but these should not be any understood as inventions and also limit the scope of what can be charged, but rather a description of the features that can be specific to particular embodiments of the particular invention it should be understood. The specific features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. On the other hand, it is possible in a variety of technical features in the context of a single embodiment also implemented separately from or in Fig plurality of embodiments in any suitable sub-combination. Further, characterized in that, but can be described as charge, such as that the operation and initially with particular combinations, one or more features from a claimed combination they can be excluded from the combination in some cases, that the claimed combination is subcombination or it may be modified by modification of the sub-combination.

Similarly, although depicted in the drawings in a particular order of operation, which handageona be carried out such operations as the specific order or sequential order shown to achieve the desired result and should not be understood to be carried out by all illustrated acts. In certain cases, it may be advantageous are multi-tasking and parallel processing. In addition, the separation of various system components in the embodiments described above are not to be understood such a separation to be required in all embodiments, the program components and systems described above are generally to be integrated or packaged into multiple software products together in a single software product it can be understood that.

Thus, persons skilled in the art will appreciate that the present invention without changing the technical spirit or essential features may be embodied in other specific forms. Therefore, the embodiment described in the above examples should be understood as illustrative and not be limiting in all aspects. The scope of the invention is intended to be included within the scope of the above description becomes than indicated by the claims, which will be described later, the spirit and scope, and all such modifications as derived from the equivalent concept of the appended claims the invention do.

The present invention can be objectively detecting and correcting the error of the entire depth map of the image through the image processing for automatic three-dimensional image transformation, by the transformation using the corrected depth map of the two-dimensional image into a three-dimensional video image It can be applied to a three-dimensional image conversion device and method using the same and the depth map generation apparatus and method capable of minimizing the error in the conversion.

Claims (15)

  1. Characteristic for extracting at least one property information for the input image information extracting unit;
    Depth map initialization unit configured to generate an initial depth map for the input image based on the characteristic information;
    FFT conversion section that performs FFT (Fast Fourier Transform) to transform into the frequency image relative to the input image; And
    Using the average value of the representative value and the initial depth map for the image frequency to obtain a correlation value, the unit determines the depth map to determine the final depth map on the basis of the correlation value;
    Depth map generation apparatus including a.
  2. According to claim 1,
    Depth, characterized in that to extract the characteristic information including the characteristic data extraction unit boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, histogram at least one of (histogram) information map generating device.
  3. According to claim 1,
    The depth map initialization unit and then separate the pixels (pixel) a plurality of constituting the input image into at least one block (block) the initial depth of the initial depth map by setting (initial depth) value for the at least one block of ( depth map generation apparatus, characterized in that for generating a depth map).
  4. According to claim 1,
    After the depth map determination section calculated the average of the depth values ​​corresponding to a block area that matches the area obtained for the representative value to obtain a representative value of the combined pixel values ​​corresponding to the high frequency area in the frequency image, in the initial depth map, depth map generation apparatus, characterized in that to obtain the correlation values ​​using the representative values ​​and the mean value.
  5. The method of claim 4, wherein
    Depth map generation apparatus, characterized in that by using the equation of the depth map to the determination unit to obtain the correlation value (CRV (Co-Relation Value)).
    Formula;
    CRV = Σ (FFT (n) * Depth (n))
    Here, the FFT (n) is the average value of the initial depth map corresponding to a block area that matches the block area of ​​a single representative, the Depth (n) is FFT (n) representing the block definition of the frequency image, n are each It means an index of a block (block).
  6. According to claim 1,
    If there is more than the depth map determination section wherein the correlation value is a predetermined threshold value, when determining the initial depth map to a final depth map, and is not the threshold value or more by inverting the depth value of the initial depth map for the inverted depth map depth map generation apparatus, characterized in that to determine the final depth map.
  7. Analyzing the input image of the two-dimensional image analysis unit to extract at least one attribute information;
    And then based on the attribute information creating an initial depth map for the input image, and converted into a frequency image by performing FFT on the input image, by using the average value of the representative value and the initial depth map of the frequency image obtaining a correlation value, setting the depth map unit for determining a final depth map on the basis of the correlation value; And
    Three-dimensional image generating unit for using the final depth map converting the input image into a three-dimensional image of the three-dimensional;
    Three-dimensional video conversion device comprising a.
  8. The method of claim 7,
    The depth map setting section,
    Depth map initialization unit configured to generate an initial depth map for the input image based on the characteristic information;
    FFT converting section for performing the FFT to transform the image frequency relative to the input image; And
    Three-dimensional video conversion device comprising a representative value of the frequency image and the using the mean value of the initial depth map to obtain the correlation value, the portion determines the depth map to determine the final depth map on the basis of the correlation value.
  9. A method for the depth map generation apparatus generating a depth map,
    Extracting at least one feature information about the input image;
    Generating an initial depth map for the input image based on the characteristic information;
    Performing the FFT to transform the image frequency relative to the input image;
    Obtaining a correlation value by using the average value of the representative value and the initial depth map of said image frequency; And
    Determining a final depth map on the basis of the correlation value;
    How to create a depth map that includes.
  10. 10. The method of claim 9,
    Obtaining a correlation value by using the average value of the representative value and the initial depth map for the image frequency, the
    Obtaining a representative value of the combined pixel values ​​corresponding to the high frequency area in the frequency image;
    The initial step in the depth map to obtain the average of the depth values ​​corresponding to a block area that matches the area obtained by the representative value; And
    How to generate a depth map comprising the step of obtaining the representative value and the calculated correlation value using the average value.
  11. 10. The method of claim 9,
    Determining a final depth map on the basis of the correlation values,
    Determining if the correlation value is not less than a predetermined threshold, if the initial depth map, not determined by the final depth map, and the threshold value or more by inverting the depth value of the initial depth map for the inverted depth map to a final depth map how to create a depth map features.
  12. Extracting at least one feature information about the input image;
    Generating an initial depth map for the input image based on the characteristic information;
    Performing the FFT to transform the image frequency relative to the input image;
    Obtaining a correlation value by using the average value of the representative value and the initial depth map of said image frequency;
    The correlation between the depth map generation determining a final depth map based on the value being recorded in the program recording medium readable in the electronic device.
  13. In the method the three-dimensional video conversion device is to automatically convert the three-dimensional image, the
    By analyzing the inputted two-dimensional image input step of extracting at least one attribute information;
    Further comprising: based on the attribute information creating an initial depth map for the input image, and determines the final depth map by checking whether the validity of the original depth map; And
    The method comprising using the final depth map converting the input image into a three-dimensional image of the three-dimensional;
    Three-dimensional image conversion method comprising a.
  14. 14. The method of claim 13,
    Determining the final depth map,
    Generating an initial depth map for the input image based on the characteristic information;
    Performing the FFT to transform the image frequency relative to the input image;
    Obtaining a representative value of the combined pixel values ​​corresponding to the high frequency area in the frequency image;
    The initial step in the depth map to obtain the average of the depth values ​​corresponding to a block area that matches the area obtained by the representative value;
    Obtaining a correlation value by using the average value of the representative value and the initial depth map of said image frequency; And
    Determining if the correlation value is not less than a predetermined threshold, if the initial depth map, not determined by the final depth map, and the threshold value or more by inverting the depth value of the initial depth map for the inverted depth map to a final depth map three-dimensional image conversion method characterized in that comprises the step of.
  15. By analyzing the inputted two-dimensional image input step of extracting at least one attribute information;
    Further comprising: based on the attribute information creating an initial depth map for the input image, and determines the final depth map by checking whether the validity of the original depth map; And
    The final depth map using a three-dimensional image conversion method comprising the step of converting the input image into a stereoscopic image of the 3D program is recorded in a recording medium readable in the electronic device.
PCT/KR2012/003347 2011-11-24 2012-04-30 Device and method for depth map generation and device and method using same for 3d image conversion WO2013077508A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR10-2011-0123374 2011-11-24
KR20110123374A KR101660808B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that

Publications (1)

Publication Number Publication Date
WO2013077508A1 true true WO2013077508A1 (en) 2013-05-30

Family

ID=48469924

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/003347 WO2013077508A1 (en) 2011-11-24 2012-04-30 Device and method for depth map generation and device and method using same for 3d image conversion

Country Status (2)

Country Link
KR (1) KR101660808B1 (en)
WO (1) WO2013077508A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20160095515A (en) * 2015-02-03 2016-08-11 연세대학교 산학협력단 3 dimensional optical measurement of defect distribution

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100019804A (en) * 2008-08-11 2010-02-19 한국전자통신연구원 Stereo vision system and control method thereof
KR20100062360A (en) * 2008-12-02 2010-06-10 삼성전자주식회사 Device and method for depth estimation
KR20100064196A (en) * 2008-12-04 2010-06-14 삼성전자주식회사 Method and appratus for estimating depth, and method and apparatus for converting 2d video to 3d video

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100873638B1 (en) * 2007-01-16 2008-12-12 삼성전자주식회사 Image processing method and apparatus
KR20100008677A (en) 2008-07-16 2010-01-26 광주과학기술원 Device and method for estimating death map, method for making intermediate view and encoding multi-view using the same

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100019804A (en) * 2008-08-11 2010-02-19 한국전자통신연구원 Stereo vision system and control method thereof
KR20100062360A (en) * 2008-12-02 2010-06-10 삼성전자주식회사 Device and method for depth estimation
KR20100064196A (en) * 2008-12-04 2010-06-14 삼성전자주식회사 Method and appratus for estimating depth, and method and apparatus for converting 2d video to 3d video

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20160095515A (en) * 2015-02-03 2016-08-11 연세대학교 산학협력단 3 dimensional optical measurement of defect distribution
KR101700109B1 (en) * 2015-02-03 2017-02-13 연세대학교 산학협력단 3 dimensional optical measurement of defect distribution

Also Published As

Publication number Publication date Type
KR101660808B1 (en) 2016-09-29 grant
KR20130057586A (en) 2013-06-03 application

Similar Documents

Publication Publication Date Title
Tang et al. Investigating haze-relevant features in a learning framework for image dehazing
US20120147139A1 (en) Stereoscopic image aligning apparatus, stereoscopic image aligning method, and program of the same
US20100238267A1 (en) System and method for combining text with three dimensional content
US20110298898A1 (en) Three dimensional image generating system and method accomodating multi-view imaging
US20110141237A1 (en) Depth map generation for a video conversion system
US20110026808A1 (en) Apparatus, method and computer-readable medium generating depth map
US20110043540A1 (en) System and method for region classification of 2d images for 2d-to-3d conversion
US20120176481A1 (en) Processing image data from multiple cameras for motion pictures
US20130250053A1 (en) System and method for real time 2d to 3d conversion of video in a digital camera
WO2008139351A1 (en) Method, apparatus and system for processing depth-related information
US20110193860A1 (en) Method and Apparatus for Converting an Overlay Area into a 3D Image
US20120262549A1 (en) Full Reference System For Predicting Subjective Quality Of Three-Dimensional Video
US8208048B2 (en) Method for high dynamic range imaging
US20080170067A1 (en) Image processing method and apparatus
KR100560464B1 (en) Multi-view display system with viewpoint adaptation
Rotem et al. Automatic video to stereoscopic video conversion
US20120113219A1 (en) Image conversion apparatus and display apparatus and methods using the same
US20130162768A1 (en) System for converting 2d video into 3d video
CN105049718A (en) Image processing method and terminal
US20080158340A1 (en) Video chat apparatus and method
CN102098528A (en) Method and device for converting planar image into stereoscopic image
Lin et al. High dynamic range imaging for stereoscopic scene representation
US20110293170A1 (en) Image processing apparatus and mathod
US20120307023A1 (en) Disparity distribution estimation for 3d tv
US20150163478A1 (en) Selecting Camera Pairs for Stereoscopic Imaging

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12852318

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12852318

Country of ref document: EP

Kind code of ref document: A1