KR101660808B1 - Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that - Google Patents

Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that Download PDF

Info

Publication number
KR101660808B1
KR101660808B1 KR1020110123374A KR20110123374A KR101660808B1 KR 101660808 B1 KR101660808 B1 KR 101660808B1 KR 1020110123374 A KR1020110123374 A KR 1020110123374A KR 20110123374 A KR20110123374 A KR 20110123374A KR 101660808 B1 KR101660808 B1 KR 101660808B1
Authority
KR
South Korea
Prior art keywords
depth map
image
value
input image
initial
Prior art date
Application number
KR1020110123374A
Other languages
Korean (ko)
Other versions
KR20130057586A (en
Inventor
우대식
김종대
박재범
전병기
정원석
Original Assignee
에스케이플래닛 주식회사
시모스 미디어텍(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 에스케이플래닛 주식회사, 시모스 미디어텍(주) filed Critical 에스케이플래닛 주식회사
Priority to KR1020110123374A priority Critical patent/KR101660808B1/en
Publication of KR20130057586A publication Critical patent/KR20130057586A/en
Application granted granted Critical
Publication of KR101660808B1 publication Critical patent/KR101660808B1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/128Adjusting depth or disparity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Abstract

The present invention relates to a depth map generating apparatus and method, and a stereoscopic image converting apparatus and method using the apparatus and method. More particularly, the present invention relates to a depth information generating apparatus and method, A depth map initialization unit for generating an initial depth map, an FFT transform unit for performing an FFT on the input image to convert the input image into a frequency image, a correlation value using a representative value of the frequency image and an average value of the initial depth map, And a depth map determining unit for determining a final depth map based on the correlation value. A depth map generating device is provided
According to the present invention, it is possible to correct an error of a depth map for a stereoscopic expression generated in an automatic stereoscopic conversion process of a video object.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a depth map generating apparatus and method, and a stereoscopic image converting apparatus and method using the apparatus and method.

More particularly, the present invention relates to a depth map generating apparatus and method, and a stereoscopic image converting apparatus and method using the depth map generating apparatus and method. More particularly, the present invention relates to a depth map generating apparatus and method, And generating an initial depth map; performing an FFT (Fast Fourier Transform) on the input image to convert the input image into a frequency image; obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map; A depth map generating apparatus and method for determining a final depth map based on the correlation value, and a stereoscopic image converting apparatus and method using the same.

Recently, interest in stereoscopic images has been amplified, and research on 3D images has been actively conducted.

Generally, it is known that humans feel the most stereoscopic effect due to the parallax between the eyes. Therefore, the 3D image can be implemented using these characteristics of the human being. For example, a specific subject is distinguished from a left eye image viewed through the left eye of the viewer and a right eye image viewed through the right eye of the viewer, and the left eye image and the right eye image are displayed at the same time, Can be seen. As a result, the 3D image can be realized by producing a binocular image divided into a left eye image and a right eye image, and displaying the binocular image.

In order to convert a monocular 2D image without depth information into a 3D image, it is necessary to render the 2D image by adding depth information.

In general, stereo conversion is divided into a manual mode and an automatic mode. The manual method is to create a depth map by watching the video according to the subjective judgment of the person about all the video literally. This process is based on the subjective judgment of a person who is able to predict the depth map even to the detail of the video while watching the video. Therefore, a person manually creates a depth map for each video object, and the error of the depth map is actually very small. However, much time and effort are required because each person directly intervenes to create a depth map of the video.

Auto-stereoscopic conversion means extracting an appropriate depth map by analyzing the characteristics of an image and generating left and right stereoscopic images using the extracted depth map. In this process, since the video object itself does not have information on the depth map, the depth map is generated by using conventional image features such as edge characteristics, color, brightness characteristics, and vanishing point characteristics of the image. However, these features do not coincide with the stereoscopic characteristics of the image of the video itself, so the stereoscopic effect is not significant.

In addition, it is impossible to extract the depth map through the image processing for each video content, and the depth map obtained through the image processing also contains many errors.

The error of the depth map obtained through this image processing can be roughly divided into two types.

One is the error or inversion phenomenon of the depth map in the partial region of the video and its combination, and the other is the error or reversal phenomenon of the depth map of the video object. Of course, such depth map errors are not easy to distinguish by technical means such as image processing.

Therefore, there is a need for a technique that can automatically detect errors in the depth map from an objective point of view only with video objects.

Korean Patent Application No. 2010-0008677 (2010.01.26), Title of the invention: depth map autonomic method and method, intermediate image generation method using the same, and encoding method of multi-view video

It is an object of the present invention to provide a depth map generating apparatus capable of correcting an error of a depth map for a stereoscopic representation generated in an automatic stereoscopic conversion process of a video object, And a stereoscopic image converting apparatus and method using the same.

It is another object of the present invention to provide a depth map generating apparatus and method capable of objectively detecting and correcting an error of a depth map of an image generated through image processing in an automatic stereoscopic image conversion, and a stereoscopic image converting apparatus and method using the same have.

It is still another object of the present invention to provide a depth map generating apparatus and method capable of correcting an error of a depth map, converting a two-dimensional image into a three-dimensional image using the corrected depth map, And a stereoscopic image converting apparatus and method using the same.

According to an aspect of the present invention, there is provided an image processing apparatus including a characteristic information extracting unit for extracting at least one characteristic information for an input image, a depth map initializing unit for generating an initial depth map for the input image based on the characteristic information, (FFT) unit for performing FFT (Fast Fourier Transform) on the frequency image to convert the frequency image into a frequency image, a correlation value is obtained using a representative value of the frequency image and an average value of the initial depth map, There is provided a depth map generating apparatus including a depth map determining section for determining a depth map.

The characteristic information extracting unit extracts characteristic information including at least one of edge information, color information, luminance information, motion information, and histogram information.

The depth map initialization unit divides a plurality of pixels constituting the input image into at least one block, sets an initial depth for the at least one block, depth map.

Wherein the depth map determining unit obtains a representative value by summing pixel values corresponding to a high frequency region in the frequency image and obtains an average value of depth values corresponding to a block region coinciding with the representative value obtained from the initial depth map, A correlation value is obtained using the representative value and the average value.

The depth map determining unit obtains a correlation value (CRV (Co-Relation Value)) using the following equation.

[Mathematical Expression]

CRV =? (FFT (n) * Depth (n))

Here, the FFT (n) is a representative value representing the block sharpness of the frequency image, the Depth (n) is an average value of the initial depth map corresponding to the block region coinciding with the block region of the FFT (n) Is the index of each block.

The depth map determining unit may determine the initial depth map as a final depth map when the correlation value is equal to or greater than a predetermined threshold and inverts the depth values of the initial depth map when the correlation value is not greater than the threshold, The map is determined as the final depth map.

According to another aspect of the present invention, there is provided an image processing apparatus including an image analysis unit for analyzing a two-dimensional input image to extract at least one characteristic information, an initial depth map for the input image based on the characteristic information, A depth map setting unit for obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map and determining a final depth map based on the correlation value, And a stereoscopic image generation unit for converting the input image into a three-dimensional stereoscopic image using the final depth map.

The depth map setting unit includes a depth map initialization unit for generating an initial depth map for the input image based on the characteristic information, an FFT transform unit for performing FFT on the input image to convert the input image into a frequency image, And a depth map determining unit for determining a final depth map based on the correlation value by using a representative value and an average value of the initial depth map.

According to another aspect of the present invention, there is provided a depth map generating apparatus for generating a depth map, comprising: extracting at least one characteristic information for an input image; Calculating a correlation value using a representative value of the frequency image and an average value of the initial depth map, calculating a correlation value using the average value of the initial depth map, And determining a final depth map based on the depth map.

Wherein the step of obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map comprises: obtaining a representative value by summing pixel values corresponding to a high frequency region in the frequency image; Obtaining a mean value of depth values corresponding to a block region coinciding with a region obtained by calculating a correlation value using the obtained representative value and the average value;

Wherein the determining of the final depth map based on the correlation value comprises determining the initial depth map as a final depth map if the correlation value is greater than or equal to a predetermined threshold, The depth values are reversed and the inverted depth map is determined as the final depth map.

According to another aspect of the present invention, there is provided an image processing method including the steps of: extracting at least one characteristic information for an input image; generating an initial depth map for the input image based on the characteristic information; Determining a final depth map based on the correlation value; and generating a depth map based on the correlation value, wherein the depth map includes at least one of: A creation method is recorded by a program, and a recording medium readable by an electronic apparatus is provided.

According to another aspect of the present invention, there is provided a method of automatically converting a stereoscopic image by a stereoscopic image conversion apparatus, comprising: extracting at least one characteristic information by analyzing an input two- Generating an initial depth map for the input image, determining whether the initial depth map is valid and determining a final depth map, and converting the input image into a three-dimensional image using the final depth map A stereoscopic image conversion method is provided.

The step of determining the final depth map may include generating an initial depth map for the input image based on the characteristic information, transforming the input image into a frequency image by performing an FFT on the input image, Obtaining a representative value by summing pixel values corresponding to a region of the frequency image, obtaining an average value of depth values corresponding to a block region coinciding with the region in which the representative value is obtained in the initial depth map, Determining an initial depth map as a final depth map if the correlation value is equal to or greater than a predetermined threshold value and determining a depth value of the initial depth map as a final depth map if the correlation value is not greater than the threshold value, And determining the inverted depth map as a final depth map.

According to another aspect of the present invention, there is provided an image processing method including extracting at least one characteristic information by analyzing an input two-dimensional input image, generating an initial depth map for the input image based on the characteristic information, Determining a final depth map by inspecting the validity of the map, and converting the input image into a three-dimensional image using the final depth map, the method comprising: A readable recording medium is provided.

Therefore, according to the present invention, it is possible to correct an error of a depth map for a stereoscopic expression generated in an automatic stereoscopic conversion process of a video object.

In addition, it is possible to objectively detect and correct an error of a depth map of an image generated through an image processing in an automatic stereoscopic image conversion.

In addition, the error of the depth map can be corrected, and a two-dimensional image can be converted into a three-dimensional image using the corrected depth map, thereby minimizing errors in image conversion.

1 is a block diagram showing a configuration of a stereoscopic image conversion apparatus according to the present invention.
2 is a block diagram schematically showing the configuration of a depth map generating apparatus according to the present invention.
3 is an exemplary view for explaining a difference between an initial depth map and a final depth map when an input image according to the present invention is converted into a three-dimensional image.
4 is a diagram illustrating an image of an FFT high-frequency component according to the present invention.
5 is an exemplary block diagram of an 8x8 block according to the present invention expressed in a frequency domain through an FFT;
6 is a diagram illustrating a method of converting a two-dimensional input image into a three-dimensional image by a three-dimensional image conversion apparatus according to the present invention.
FIG. 7 is a flowchart illustrating a method of generating a depth map by a depth map generating apparatus according to the present invention. FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description with reference to the accompanying drawings, the same or corresponding components will be denoted by the same reference numerals, and redundant description thereof will be omitted.

Hereinafter, a depth map expressing a three-dimensional object is created and corrected for a video object satisfying the following two precondition conditions.

 Prerequisite 1 is "usually a bright color or a strong brightness is often close to the distance". For example, the farther a mountain is, the more likely it is that the color is converted to an achromatic color compared to a nearby mountain. Or, the farther away a bright, strong-colored object is, the less saturated the color will be. This characteristic is also a feature of a camera that captures a person's visual characteristics.

The precondition 2 is that "usually, a clear part in an image is close in distance ". For example, when the same object is close and when it is far away, the sharpness of the object is different. This occurs visually or because of the limitations of the camera's resolution.

The present invention utilizes the above-mentioned two characteristics to transform a three-dimensional object and correct it. The precondition 1 is a condition for converting a solid, and the precondition 2 is a means for correcting the condition. In other words, the depth map is extracted with the color and brightness information of the video object, and the depth map is error-detected and corrected by objectively verifying whether the depth map is partially or totally valid.

1 is a block diagram showing a configuration of a stereoscopic image conversion apparatus according to the present invention.

Referring to FIG. 1, a stereoscopic image conversion apparatus 100 includes an image analysis unit 110, a depth map setting unit 120, and a stereoscopic image generation unit 130.

The image analyzer 110 analyzes the two-dimensional input image and extracts at least one characteristic information. The characteristic information includes edge information, color information, luminance information, motion information, histogram information, and the like.

The depth map setting unit 120 generates an initial depth map for the input image based on the feature information extracted by the image analysis unit 110 and performs Fast Fourier Transform (FFT) on the input image Frequency image, a correlation value is obtained using a representative value of the frequency image and an average value of the initial depth map, and a final depth map is determined based on the correlation value.

The depth map setting unit 120 will be described in detail with reference to FIG.

The stereoscopic image generator 130 converts the two-dimensional input image into a three-dimensional stereoscopic image using the final depth map determined by the depth map setting unit 120. For example, the stereoscopic image generating unit 130 may generate parallax information using the final depth map, and may generate a three-dimensional stereoscopic image using the parallax information. The three-dimensional stereoscopic images generated in this way appear more stereoscopic as the depth values for each pixel in each frame vary.

Here, the stereoscopic image generation unit 130 converts the two-dimensional image into the three-dimensional image using the time difference information. However, the stereoscopic image generation unit 130 may generate the input image using the final depth map A method of converting into a stereoscopic image follows various conventional methods.

The stereoscopic image conversion apparatus 100 configured as described above can convert a 2D input image into a stereoscopic video by setting a depth value of the input image based on characteristic information of the input image.

FIG. 2 is a block diagram schematically showing a configuration of a depth map generating apparatus according to the present invention. FIG. 3 is a diagram illustrating a difference between an initial depth map and a final depth map when converting an input image according to the present invention into a three- FIG. 4 is a diagram illustrating an image of an FFT high-frequency component according to the present invention, and FIG. 5 is an exemplary view of a block representing an 8 × 8 block according to the present invention in a frequency domain through an FFT.

In FIG. 1, the depth map setting unit has been described. In FIG. 2, the depth map generating apparatus 200 will be described.

2, the depth map generating apparatus 200 includes a feature information extracting unit 210, a depth map initializing unit 220, a fast Fourier transform (FFT) converting unit 230, and a depth map determining unit 240 .

The characteristic information extracting unit 210 extracts at least one characteristic information for the input image. Here, the input image may be a monocular image.

The characteristic information extracted by the characteristic information extracting unit 210 may be edge information, color information, luminance information, motion information, or histogram information.

The characteristic information extracting unit 210 extracts characteristic information in an image through various analysis methods on a pixel or block basis to collect basic information for depth map generation.

The depth map initialization unit 220 generates an initial depth map for the input image based on the characteristic information extracted by the characteristic information extraction unit 210.

That is, the depth map initialization unit 220 divides a plurality of pixels constituting the input image into at least one block, and then stores an initial depth value for the at least one block And creates an initial depth map.

The depth map initialization unit 220 generates a depth map for each frame of the two-dimensional image based on the extracted characteristic information. That is, the depth map initialization unit 220 extracts depth values for each pixel from each depth map of the two-dimensional image. Here, the depth map is a data structure storing depth values of each pixel per frame for a two-dimensional image.

An example in which an input image is converted into a three-dimensional image using an initial depth map generated by the depth map initialization unit 220 will be described with reference to FIG.

Looking at the image of Figure 3a, the spider is at the front, followed by the flowers and the green background. Therefore, if the image of FIG. 3A is expressed in three-dimensional form, it should be able to be separated into three stages as appropriate, such as spider, flower, and background. Of course, it is possible to express more precisely, but as an example for explanation, it is divided into three stages. This can be done with subjective visual characteristics, but if you want to automatically analyze it through image processing, you can use the precondition 1 to determine the most distant spider is achromatic and dark, It will be an expression.

If the image of FIG. 3A is converted into a stereoscopic image by the depth map according to the precondition 1, the stereoscopic image can not be expressed normally as shown in FIG. 3b.

However, if the depth map generated in the depth map initialization unit 220 is changed to a film image reversed like a print film of a photograph, a normal depth map can be formed.

That is, if the initial depth map generated as shown in FIG. 3B can be converted into a depth map as shown in FIG. 3C, a less error-free stereoscopic display is possible.

In the image of FIG. 3c, the spider is on the front side, so the spider is brightly expressed, then the flower region, and the background is darkened because there is almost no change in the background. As a result, if the depth map of FIG. 3C is selected, it is possible to normalize three-dimensional objects while improving errors.

Therefore, the depth map generation apparatus 200 should determine whether to convert the image of FIG. 3A into a depth map of FIG. 3B or to select a depth map of FIG. 3C and convert the image.

The depth map generator 200 performs an FFT (Fast Fourier Transform) transformation on the input image to determine whether to select the depth map of FIG. 3B or to convert the depth map of FIG. 3C into the depth map of FIG. And the final depth map is determined using the frequency image.

Accordingly, the depth map generator 200 includes an FFT transformer 230 and a depth map determiner 240.

The FFT transformer 230 performs an FFT on the input image to convert the input image into a frequency-domain frequency image. That is, the FFT transformer 230 transforms the input image in the space into the frequency image.

Referring to FIG. 4, an image obtained by performing the FFT on the FFT transform unit 230 and transforming the input image into a frequency image will be referred to. Referring to FIG. 4, the bright part has a clear or strong outer edge. That is, if the high frequency region in the frequency domain, that is, the portion expressing the sharpness or the strong outer angle is expressed in bright color, the outer portion of the spider is brightly expressed in the image of FIG. 3A, It becomes dark.

FIG. 5A shows an arbitrary area of 8x8 pixels in the input image, and FIG. 5B shows the frequency domain of FIG. 5A through the FFT.

In Fig. 5 (b), the A region refers to a low frequency band and the B region refers to a high frequency band. That is, the B region is a high frequency component. If the image has a clear or strong outer edge, the value of the B region becomes larger. The simpler the value of the A region becomes, the smaller the value of the B region becomes.

In this manner, the total sum of the pixel values belonging to the high frequency region in the frequency image, i.e., the sum of the B regions expressing the sharpness in the 8x8 block belonging to the partial region of the image and representing the large deviation between the pixels of the image, pixel is defined as a representative value corresponding to pixel. In this manner, if the magnitude value of each block unit is expressed as a brightness value, the image is expressed using an FFT high frequency component as shown in FIG.

The depth map determining unit 240 obtains a correlation value using a representative value of the frequency image generated by the FFT transforming unit 230 and an average value of the initial depth map, .

That is, the depth map determining unit 240 obtains a representative value (FFT (n)) expressing the block sharpness in units of 8x8 blocks in the frequency image formed through the FFT transforming unit 230. Here, the representative value is obtained by summing pixel values corresponding to a high frequency region in the frequency image. Then, the depth map determining unit 240 obtains an average value Depth (n) of the 8x8 block of the initial depth map coinciding with the block region of the FFT (n) Value (CRV (Co-Relation Value)).

The depth map determination unit 240 obtains a correlation value (CRV (Co-Relation Value)) using Equation (1).

Figure 112011093126153-pat00001

Herein, the FFT (n) is a representative value representing the block sharpness of the frequency image, the Depth (n) is an average value of the initial depth map corresponding to the block region coinciding with the block region of the FFT (n) Means the index of each block. Wherein the FFT (n) refers to a sum of pixel values corresponding to a high frequency region in an FFT-transformed frequency image, and the average value of the initial depth map corresponds to a block region coinciding with a region in which the representative value is obtained in the initial depth map Means the average value of the depth values.

When the correlation value is obtained through Equation 1, the depth map determining unit 240 determines the initial depth map as the final depth map when the correlation value is equal to or greater than the predetermined threshold, The depth values of the initial depth map are reversed and the inverted depth map is determined as the final depth map.

As a result, the depth map determining unit 240 uses the frequency image converted into the image of the frequency domain as shown in FIG. 4 as a depth map for automatically transforming the image of FIG. 3A into the depth map of FIG. 3C This is a more valid conclusion.

The depth map determination unit 240 determines whether the condition that the color of the precondition 1 is strong and the bright color precedes and the condition that the clear portion is in front is matched, that is, when the correlation value is equal to or greater than a predetermined threshold, The depth map of Fig. 3B generated as " 1 "

If the correlation value is not equal to or greater than the predetermined threshold value, the depth map determining unit 240 determines that the depth map reflecting the reversal phenomenon is more valid as shown in FIG. 3C.

The depth map generator 200 configured as described above extracts an initial depth map using characteristic information such as color and brightness information of a video object and performs objective verification of whether the initial depth map is partially or wholly valid, An error is detected and a final depth map is generated.

6 is a diagram illustrating a method for converting a two-dimensional input image into a three-dimensional image by a three-dimensional image conversion apparatus according to the present invention.

Referring to FIG. 6, the stereoscopic image conversion apparatus analyzes a two-dimensional input image to extract at least one characteristic information (S602). Here, the characteristic information includes edge information, color information, luminance information, motion information, histogram information, and the like.

After performing step S602, the stereoscopic image converting apparatus generates an initial depth map for the input image based on the characteristic information (S604).

Then, the stereoscopic image converting apparatus determines the final depth map by objectively verifying whether the initial depth map is partially or wholly valid (S606).

That is, the stereoscopic image conversion apparatus performs FFT on the input image to convert the input image into a frequency image. Then, the stereoscopic image conversion apparatus obtains a representative value by summing pixel values belonging to a high frequency region in the frequency image, and obtains an average value of depth values corresponding to a block region coinciding with the representative value in the initial depth map. Then, the stereoscopic image conversion apparatus obtains a correlation value using Equation (1) using the obtained representative value and the average value.

Then, the stereoscopic image converting apparatus determines a final depth map based on the correlation value.

After the execution of S606, the stereoscopic image conversion apparatus converts the input image into a three-dimensional image using the determined final depth map (S608).

FIG. 7 is a flowchart illustrating a method of generating a depth map by a depth map generating apparatus according to the present invention.

Referring to FIG. 7, the depth map generation apparatus extracts at least one characteristic information about an input image, and generates an initial depth map for the input image based on the extracted characteristic information (S702). That is, the depth map generation apparatus extracts characteristic information such as edge information, color information, luminance information, motion information, and histogram information. Then, the depth map generating apparatus divides a plurality of pixels constituting the input image into at least one block, sets an initial depth for the at least one block, Creates a depth map.

After performing step S702, the depth map generator converts the input image into a frequency image by performing an FFT on the input image (step S704).

After step S704, the depth map generator determines whether the transformed frequency image satisfies the precondition (S706). Here, the precondition is the precondition 2, and the precondition 2 is "the clear part in the image is usually close in distance ".

If it is determined in step S706 that the image satisfies the prerequisite condition, the depth map generator obtains a correlation value using a representative value of the frequency image and an average value of the initial depth map (S708). A detailed description of the method for obtaining the correlation value will be given with reference to FIG.

After step S708, the depth map generator determines whether the correlation value is greater than or equal to a predetermined threshold (S710).

If it is determined in step S710 that the depth map is greater than or equal to the threshold, the depth map generation apparatus determines the initial depth map as the final depth map (step S712).

If it is determined in step S710 that the correlation value is not equal to or greater than the threshold value, the depth map generator inverts depth values of the initial depth map (step S714) (S716).

Thus, those skilled in the art will appreciate that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the embodiments described above are to be considered in all respects only as illustrative and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

The present invention can objectively detect and correct an error of an overall depth map of an image through an image processing in an automatic stereoscopic image conversion, convert a two-dimensional image into a three-dimensional image using the corrected depth map, The present invention can be applied to a depth map generating apparatus and method capable of minimizing errors in conversion, and a stereoscopic image converting apparatus and method using the same.

100: stereoscopic image conversion apparatus 210: image analysis unit
220: depth map setting unit 230: stereoscopic image generating unit
200: depth map generator 210: characteristic information extractor
220: depth map initialization unit 230: FFT transform unit
240: Depth map determining unit

Claims (15)

A characteristic information extracting unit for extracting at least one characteristic information for an input image;
A depth map initialization unit for generating an initial depth map for the input image based on the characteristic information;
An FFT transformer for transforming the input image into a frequency image by performing FFT (Fast Fourier Transform); And
A depth map determining unit that obtains a correlation value using a representative value of the frequency image and an average value of the initial depth map, and determines a final depth map based on the correlation value;
And a depth map generation unit for generating a depth map.
The method according to claim 1,
Wherein the characteristic information extraction unit extracts characteristic information including at least one of edge information, color information, luminance information, motion information, and histogram information, Map generator.
The method according to claim 1,
The depth map initialization unit divides a plurality of pixels constituting the input image into at least one block, sets an initial depth for the at least one block, depth map of the depth map.
The method according to claim 1,
Wherein the depth map determining unit obtains a representative value by summing pixel values corresponding to a high frequency region in the frequency image and obtains an average value of depth values corresponding to a block region coinciding with the representative value obtained from the initial depth map, And a correlation value is obtained by using the representative value and the average value.
The method of claim 4, wherein
Wherein the depth map determination unit obtains a correlation value (CRV (Co-Relation Value)) using the following equation.
[Mathematical Expression]
CRV =? (FFT (n) * Depth (n))
Here, the FFT (n) is a representative value representing the block sharpness of the frequency image, the Depth (n) is an average value of an initial depth map corresponding to a block region coinciding with a block region of FFT (n) Indicates the index of the block.
The method according to claim 1,
Wherein the depth map determining unit determines the initial depth map as a final depth map when the correlation value is equal to or greater than a predetermined threshold and inverts the depth values of the initial depth map when the correlation value is not equal to or greater than the threshold, And the final depth map is determined as the final depth map.
An image analyzer for analyzing a two-dimensional input image to extract at least one characteristic information;
An initial depth map for the input image is generated based on the characteristic information, an FFT is performed on the input image to convert the input image into a frequency image, and a representative value of the frequency image and an average value of the initial depth map are used A depth map setting unit for obtaining a correlation value and determining a final depth map based on the correlation value; And
A stereoscopic image generation unit for converting the input image into a three-dimensional stereoscopic image using the final depth map;
Dimensional image.
8. The method of claim 7,
The depth map setting unit may set,
A depth map initialization unit for generating an initial depth map for the input image based on the characteristic information;
An FFT transformer for transforming the input image into a frequency image by performing FFT; And
And a depth map determination unit for determining a correlation value using a representative value of the frequency image and an average value of the initial depth map and determining a final depth map based on the correlation value.
A method of generating a depth map by a depth map generator,
Extracting at least one characteristic information for an input image;
Generating an initial depth map for the input image based on the characteristic information;
Transforming the input image into a frequency image by performing FFT;
Obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map; And
Determining a final depth map based on the correlation value;
/ RTI >
10. The method of claim 9,
Wherein the step of obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map,
Obtaining a representative value by summing pixel values corresponding to a high frequency region in the frequency image;
Obtaining an average value of depth values corresponding to a block area coinciding with the area in which the representative value is obtained in the initial depth map; And
And obtaining a correlation value using the obtained representative value and the average value.
10. The method of claim 9,
Wherein determining the final depth map based on the correlation value comprises:
Determining the initial depth map as a final depth map if the correlation value is greater than or equal to a predetermined threshold and inverting the depth values of the initial depth map if the correlation value is not greater than the threshold value and determining the inverted depth map as a final depth map And the depth map is generated.
Extracting at least one characteristic information for an input image;
Generating an initial depth map for the input image based on the characteristic information;
Transforming the input image into a frequency image by performing FFT;
Obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map;
And determining a final depth map based on the correlation value, wherein the depth map is recorded by the program.
A method for automatically converting a stereoscopic image by a stereoscopic image conversion apparatus,
Analyzing the input two-dimensional input image to extract at least one characteristic information;
An initial depth map for the input image is generated based on the characteristic information, an FFT is performed on the input image to convert the input image into a frequency image, and a representative value of the frequency image and an average value of the initial depth map are used Obtaining a correlation value, and determining a final depth map based on the correlation value; And
Transforming the input image into a three-dimensional image using the final depth map;
Dimensional image.
14. The method of claim 13,
Wherein determining the final depth map comprises:
Generating an initial depth map for the input image based on the characteristic information;
Transforming the input image into a frequency image by performing FFT;
Obtaining a representative value by summing pixel values corresponding to a high frequency region in the frequency image;
Obtaining an average value of depth values corresponding to a block area coinciding with the area in which the representative value is obtained in the initial depth map;
Obtaining a correlation value using a representative value of the frequency image and an average value of the initial depth map; And
Determining the initial depth map as a final depth map if the correlation value is greater than or equal to a predetermined threshold and inverting the depth values of the initial depth map if the correlation value is not greater than the threshold value and determining the inverted depth map as a final depth map And converting the stereoscopic image into a stereoscopic image.
delete
KR1020110123374A 2011-11-24 2011-11-24 Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that KR101660808B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020110123374A KR101660808B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020110123374A KR101660808B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that
PCT/KR2012/003347 WO2013077508A1 (en) 2011-11-24 2012-04-30 Device and method for depth map generation and device and method using same for 3d image conversion

Related Child Applications (1)

Application Number Title Priority Date Filing Date
KR1020160087455A Division KR101720161B1 (en) 2016-07-11 2016-07-11 Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that

Publications (2)

Publication Number Publication Date
KR20130057586A KR20130057586A (en) 2013-06-03
KR101660808B1 true KR101660808B1 (en) 2016-09-29

Family

ID=48469924

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020110123374A KR101660808B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that

Country Status (2)

Country Link
KR (1) KR101660808B1 (en)
WO (1) WO2013077508A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101700109B1 (en) * 2015-02-03 2017-02-13 연세대학교 산학협력단 3 dimensional optical measurement of defect distribution
CN108076338B (en) 2016-11-14 2022-04-08 北京三星通信技术研究有限公司 Image visual processing method, device and equipment

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100873638B1 (en) * 2007-01-16 2008-12-12 삼성전자주식회사 Image processing method and apparatus
KR20100008677A (en) 2008-07-16 2010-01-26 광주과학기술원 Device and method for estimating death map, method for making intermediate view and encoding multi-view using the same
KR100942925B1 (en) * 2008-08-11 2010-02-22 한국전자통신연구원 Stereo vision system and control method thereof
KR101491556B1 (en) * 2008-12-02 2015-02-09 삼성전자주식회사 Device and method for depth estimation
KR101506926B1 (en) * 2008-12-04 2015-03-30 삼성전자주식회사 Method and appratus for estimating depth, and method and apparatus for converting 2d video to 3d video

Also Published As

Publication number Publication date
KR20130057586A (en) 2013-06-03
WO2013077508A1 (en) 2013-05-30

Similar Documents

Publication Publication Date Title
KR101720161B1 (en) Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that
JP6027034B2 (en) 3D image error improving method and apparatus
US8718356B2 (en) Method and apparatus for 2D to 3D conversion using scene classification and face detection
EP2268047A2 (en) Conversion device and method converting a two dimensional image to a three dimensional image
RU2423018C2 (en) Method and system to convert stereo content
JP2013527646A5 (en)
EP3311361B1 (en) Method and apparatus for determining a depth map for an image
KR101332638B1 (en) Apparatus and method for correcting depth map and apparatus and method for generating 3d conversion image using the same
US8565513B2 (en) Image processing method for providing depth information and image processing system using the same
KR20150021522A (en) Method and device for implementing stereo imaging
JP2015015713A (en) Method and device for assessing presence of watermark in video
KR101220223B1 (en) Method and apparatus for visual discomfort metric of stereoscopic video, recordable medium which program for executing method is recorded
KR101660808B1 (en) Apparatus and Method for generating Depth Map, stereo-scopic image conversion apparatus and method usig that
KR101797035B1 (en) Method for converting overlaying area into 3D image and apparatus thereof
Azzari et al. A modified non-local mean inpainting technique for occlusion filling in depth-image-based rendering
TW201622418A (en) Processing of disparity of a three dimensional image
CN104982032B (en) The method and apparatus of 3D rendering data segmentation
Hasan et al. No-reference quality assessment of 3D videos based on human visual perception
Chang et al. Ghosting reduction method for color anaglyphs
Fan et al. On the performance of 3D just noticeable difference models
Jung et al. Detection of the Single Image from DIBR Based on 3D Warping Trace and Edge Matching
Faridul et al. Disparity estimation and disparity-coherent watermarking
Vatolin et al. Sharpness mismatch and 6 other stereoscopic artifacts measured on 10 Chinese S3D movies
Fezza et al. Universal blind image quality assessment for stereoscopic images
KR101528683B1 (en) Detecting method for excessive disparity object

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
A107 Divisional application of patent
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20190903

Year of fee payment: 4