WO2013077508A1  Device and method for depth map generation and device and method using same for 3d image conversion  Google Patents
Device and method for depth map generation and device and method using same for 3d image conversion Download PDFInfo
 Publication number
 WO2013077508A1 WO2013077508A1 PCT/KR2012/003347 KR2012003347W WO2013077508A1 WO 2013077508 A1 WO2013077508 A1 WO 2013077508A1 KR 2012003347 W KR2012003347 W KR 2012003347W WO 2013077508 A1 WO2013077508 A1 WO 2013077508A1
 Authority
 WO
 WIPO (PCT)
 Prior art keywords
 depth map
 image
 value
 initial
 input image
 Prior art date
Links
Images
Classifications

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T7/00—Image analysis
 G06T7/50—Depth or shape recovery

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N13/00—Stereoscopic video systems; Multiview video systems; Details thereof
 H04N13/10—Processing, recording or transmission of stereoscopic or multiview image signals
 H04N13/106—Processing image signals
 H04N13/128—Adjusting depth or disparity

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N13/00—Stereoscopic video systems; Multiview video systems; Details thereof
 H04N2013/0074—Stereoscopic image analysis
 H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Abstract
Description
The present invention is a depth map generation related to a threedimensional image conversion device and method using the same and an apparatus and method, and more particularly, to extract at least one characteristic information about the input image, based on the characteristic information of the input image to generate the initial depth map, after performing the FFT (Fast Fourier transform) to transform into the frequency image for the input image, by using the average value of the representative value and the initial depth map of the frequency image to obtain a correlation value, on the basis of the correlation value it relates to a threedimensional image conversion device and method using the same and the depth map generation apparatus and method to determine the final depth map.
As the recently amplified interest in the 3D image (Stereoscopic image), it has been actively studied for the 3D image.
In general, humans are known to feel the cubic effect by the time difference between both eyes most. Accordingly, 3D video may be implemented using a human such properties. For example, a particular subject, a lefteye image and the viewer's to distinguish the righteye image viewed through the right eye, the viewer has the particular subject by displaying the lefteye image and the righteye image at the same time as shown by the left eye of the viewer with a 3D image you can so that you can see. After all, 3D images can be implemented by displaying it by making the both eyes (binocular) image divided into a lefteye image and righteye images.
In order to convert depth information without a monocular (monocular) 2D image into a 3D image it is required the task of rendering (rendering) in addition to the depth information for the 2D image.
Typically, the threedimensional transformation is divided into manual mode and automatic mode. Passive way is to create a depth map So while watching the video on the subjective judgment of a person for literally any video. This process is watching the footage is based on the subjective judgment of people you can expect the depth map, even small parts of the video. Therefore, it is the direct production of a depth map for each person in the footage, errors of fact, the depth map is very small. However, a lot of time and effort is required to create a depth map because of the footage directly to the people involved in every video.
Autostereoscopic conversion means for analyzing characteristics of the image to extract the proper depth map, and generate the left and right stereoscopic image by using this. In this process, the video object itself since there is no information on the depth map utilizing conventional image features such as the outer (Edge) characteristics of the image, color, brightness characteristics, the vanishing point characteristics and generates the depth map. However, these features are not the effect of threedimensional large because in many cases does not match the characteristics of the threedimensional image with the footage itself.
And, inside a video object it will include a video content in various forms and comprises a depth map, too many errors obtained Extracting depth maps through an image processing on the contents of each video object, and practically impossible, through the image processing.
Error of the depth map obtained through such image processing can be classified into two types.
One of the error or reversal of the depth map of the partial area of the video object, and a combination thereof and the other is an error or a reversal of the entire depth map video. Of course, errors of this depth map is not quite so simple to distinguish the technical methods such as image processing.
Thus, there is a need for a technology that can detect an error in the depth map automatically from an objective point of view has only video.
The present invention been made in view of solving the above problems, an object of the present invention is a depth capable of correcting the error in the depth map (Depth Map) for a threedimensional expression generated from the autostereoscopic conversion process in the video object map generation apparatus and to provide a method and a stereoscopic image using the same transformation method and apparatus.
Another object of the present invention provides a threedimensional image conversion device and method using the same errors in the depth map of the image generated by the image processing for automatic threedimensional image transformation and objective detection and correction depth map generation apparatus and method for a have.
And It is another object of the present invention corrects the error of the depth map, and by using the corrected depth map convert the 2D image into a 3D image and to minimize the error in the image conversion depth map generation apparatus and method and it is to provide a threedimensional image conversion apparatus and method.
According to an aspect of the invention, characteristic to extract at least one characteristic information on the input image information extracting unit, unit based on the attribute information initializing the depth map to generate an initial depth map for the input image, the input image performing a FFT (Fast Fourier transform) for by using the average value of the representative value and the initial depth map for the FFT converter, the frequency image to be converted to a frequency image to obtain a correlation value, the end on the basis of the correlation values the depth map generation apparatus comprising: a decision depth map for determining a depth map is provided.
* Is the characteristic data extraction unit extracting characteristic information including the boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, at least one of a histogram (histogram) information.
The depth map initialization unit at least after separated by one block (block) the initial depth by setting the initial depth values (initial depth) for the at least one block of a map of pixels (pixel) a plurality of constituting the input image ( It generates a depth map).
After the depth map determination section calculated the average of the depth values corresponding to a block area that matches the area obtained for the representative value to obtain a representative value of the combined pixel values corresponding to the high frequency area in the frequency image, in the initial depth map, the representative by using the value and the average value is obtained for the correlation values.
The depth map determination section using the equation to calculate the correlation value (CRV (CoRelation Value)).
Formula;
CRV = Σ (FFT (n) * Depth (n))
Here, the FFT (n) is a representative value representing the block definition of the frequency image, and the Depth (n) is the average value of the initial depth map corresponding to a block area that matches the block area of the FFT (n), n is means an index for each block (block).
Further, when the depth map determination section wherein the correlation value is not less than a predetermined threshold, and determining the initial depth map to a final depth map, other than the threshold value or more by inverting the depth value of the initial depth map that the inverted depth to determine the final depth map to map.
According to another aspect of the invention, and to generate the initial depth map for the input image based on the image analysis unit, the characteristic data to extract at least one characteristic information by analyzing the input image of the twodimensional, in the input image after conversion by performing the FFT to the frequency image for setting the depth map by using the average value of the representative value and the initial depth map of the frequency image to obtain a correlation value, and determine the final depth map on the basis of the correlation values the threedimensional video conversion device including unit, by using the end parts of the threedimensional depth map image generator for converting the input image into a threedimensional image of the threedimensional is provided.
The depth map setting unit of the FFT converter, the frequency image conversion on the basis of the characteristic data to the frequency image by performing FFT on the depth map initialization unit, the input image to generate the initial depth map for the input image using the average value of the representative value and the initial depth map to obtain a correlation value, and comprising a decision depth map to determine the final depth map on the basis of the correlation value.
In accordance with another aspect of the invention, the depth map generation unit depth according to a method of generating a map comprising the steps of: extracting at least one feature information about the input image, the initial depth for the input image based on the attribute information generating a map, the method to obtain the step of performing an FFT to transform the frequency image for the input image, a correlation with a mean value of the representative value and the initial depth map of the frequency image relationship values, the correlation values the depth map generation method determining a final depth map is provided as a basis.
Obtaining a correlation value by using the average value of the representative value and the initial depth map of the frequency image, the step of obtaining the representative value of the combined pixel values corresponding to the high frequency area in the frequency image, the representative at the initial depth map and finding an average of the depth values corresponding to a block area that matches the value obtained with the area, and a step of obtaining a correlation value using the calculated representative value and the average value.
Determining a final depth map on the basis of the correlation value, if the correlation value is not less than a predetermined threshold, and determining the initial depth map to a final depth map, other than the threshold value or more of the initial depth map depth reversal values by characterizes in determining the depth map by inverting the final depth map.
In accordance with another aspect of the invention, performing FFT on the phase, the input image to generate the initial depth map for the input image on the basis of the step, the characteristic information for extracting the at least one property information for the input image the depth map comprising the step of determining the final depth map by obtaining a correlation value by using the average value of the representative value and the initial depth map of the frequency image, based on the correlation value to be converted to a frequency image generating method is recorded to the program is provided with a recording medium readable in the electronic device.
In accordance with another aspect of the invention, there is provided a method of threedimensional image converting unit converts the autostereoscopic image, the analyzes an input image of an inputted twodimensional basis of the steps, the characteristic data to extract at least one characteristic information to generate the initial depth map for the input image, and using the step, the final depth map to determine the final depth map by checking the validity whether the initial depth map for converting the input image into a threedimensional image of a threedimensional the threedimensional image conversion method comprises the step is provided.
Determining the final depth map comprises the steps of converting the frequency image by performing the FFT on the phase, the input image to generate the initial depth map for the input image based on the characteristic information, a high frequency in the frequency picture obtaining a representative value of the combined pixel values corresponding to the region, and the step from the initial depth map to obtain the average of the depth values corresponding to a block area that matches the area obtained for the representative value, representative of the frequency image value and the initial obtaining a correlation value by using the average value of the depth map, the correlation value is predetermined not less than the threshold value, the depth value of the initial depth map if the initial depth map, not determined by the final depth map, and the threshold value or more reversal of determining the inverted depth map to a final depth map.
In accordance with another aspect of the invention, by analyzing an input image of the input 2D and generate an initial depth map for the input image on the basis of the phase, the characteristics of extracting at least one feature information, the initial depth determining a final depth map by checking the validity whether the map, the threedimensional image conversion method comprising the step of using the final depth map converting the input image into a threedimensional image of the threedimensional is recorded as a program in an electronic device the readable storage medium is provided.
Therefore, according to the present invention, it is possible to correct errors of the depth map (Depth Map) for a threedimensional expression generated from the autostereoscopic conversion process in the video object.
Further, it is possible to objectively detect and correct the error in the depth map of the image generated through the autostereoscopic image conversion processing.
In addition, correct errors in the depth map, and it is possible to minimize the error of the image conversion to convert the 2D image into a 3D image using the corrected depth map.
1 is a block diagram showing the configuration of a threedimensional video conversion device according to the present invention.
2 is a block diagram schematically showing the configuration of a depth map generation unit according to the invention.
3 is a case to convert the input image according to the present invention as the stereoscopic images of threedimensional an illustration for explaining the difference between the initial depth map and the final depth map.
Figure 4 is an exemplary view illustrating an image of the FFT frequency component according to the invention.
Figure 5 is illustrative of a block diagram representation of a 8x8 block according to the present invention into a frequency domain through a FFT.
Figure 6 is a view showing how the threedimensional image conversion apparatus according to the invention converts the input image of the twodimensional threedimensional image of a threedimensional.
7 is a flowchart illustrating a method of generating a depth map generation unit depth map in accordance with the present invention.
With reference to the accompanying drawings, the present will be described in more detail a preferred embodiment of the invention. Description of the same components in the following description with reference to the accompanying drawings or corresponding are assigned the same reference numerals and a duplicate thereof will be omitted.
Hereinafter, to generate a depth map representing the threedimensional and corrected for the video object that satisfies the two prerequisites below.
1 is a prerequisite "It is typical in light color or a strong brightness is often a close distance." For example, if a lot more acid when compared to away mountains that close while viewing the picture seems to be converted into an achromatic color. Or the bright color of the strong object more far away quality when it is about saturated colors. These features are characteristic of human vision, but also is also a feature of a camera for capturing them.
Prerequisite 2 is "typically the case with many bright parts in the picture that close." For example, when the far and when the same objects are near a sharpness of an object varies This occurs due to the limitation of the resolution of the camera for visually or recording.
The present invention is directed to the converter and correcting it by using the threedimensional two properties as described above. 1 is a prerequisite condition for converting the threedimensional, two prerequisites is a means for correcting it. That is, it extracts a depth map with the color, brightness information of the video object, and detect an error in the depth map with the objective verification of such a depth map in part or entirely on whether a valid and correct them.
1 is a block diagram showing the configuration of a threedimensional video conversion device according to the present invention.
1, a threedimensional image conversion device 100 comprises an image analysis unit 110, a depth map setting unit 120, a threedimensional image generation unit 130.
The image analysis unit 110 analyzes the input image of the twodimensional and extracts at least one feature information. The characteristic information includes a boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, a histogram (histogram) information, and the like.
And the depth map setting unit 120 generates an initial depth map for the input image based on the attribute information extracted by the image analysis unit 110, and performs FFT (Fast Fourier Transform) on the input image after converting to a frequency image by using the average value of the representative value and the initial depth map for the image frequency to obtain a correlation value, and determines the final depth map on the basis of the correlation value.
A detailed description of the depth map setting unit 120 will be described with reference to Fig.
The threedimensional image generation section 130 is converted by the final depth map is determined by the setting unit 120, the depth map for the input image of the twodimensional to threedimensional image of a threedimensional. For example, the threedimensional image generation section 130 may generate the time difference (parallax) information using the final depth map, and generating a 3D stereoscopic image based on the time difference information. The generated threedimensional image appears to allow more depth (depth) value for each of the pixels to vary in each frame, more threedimensional appearance.
Here, the input to the threedimensional image generation unit 130 has been described as using the time difference information converting a twodimensional image into a threedimensional image of a threedimensional, the threedimensional image generation unit 130 is used for the final depth map image how to convert the threedimensional image are different according to prior methods.
Threedimensional image conversion device 100 configured as described above can be converted, by setting the depth values for the input image based on the characteristic information of the input image, the input 2D image of the 3D image (Stereoscopic Video).
Figure 2 explains the difference between the initial depth map and the final depth map when converting the input image according to the third is the present invention, a block diagram schematically showing the configuration of a depth map generation unit according to the invention with threedimensional image of a threedimensional for example, Fig. 4 is an exemplary view, Figure 5 is illustrative of a block diagram representation of a 8x8 block according to the present invention into a frequency domain through the FFT illustrating an image of the FFT frequency component according to the invention.
In Figure 1 has been described to depth map setting part, be described with the depth map generating unit 200 in Fig.
2, the depth map generation apparatus 200 includes a characteristic data extraction unit 210, the depth map initialization unit (220), (Fast Fourier Transform), FFT conversion unit 230, determines the depth map 240 It includes.
The characteristic data extraction unit 210 extracts at least one property information for the input image. Here, the input image can be a monocular (monocular) image.
Attribute that is the characteristic data extraction unit 210 extracts the information may be a boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, or histogram (histogram) information, and the like.
The characteristic data extraction unit 210 extracts a pixel (pixel) or the block (block) in the image attribute information through a variety of analysis methods of the unit to collect information that is the basis for the depth map generation.
The depth map initialization unit 220 generates an initial depth map for the input image based on the attribute information extracted by the characteristic data extraction unit 210.
That is, the depth map initialization unit 220 has at least one block (block) by one minutes after the initial depth value for the at least one block (initial depth), a plurality of pixels (pixel) forming the input image set to generate the initial depth map (depth map).
The depth map initialization unit 220 generates a depth map (depth map) for each frame of the twodimensional image based on the extracted characteristic information. That is, the depth map initialization unit 220 is to extract the depth value for each pixel for each frame from depth maps (depth map) for the 2D image. Here, the depth map is a data structure storing a depth value for each pixel per frame for the twodimensional image (depth value).
For example by converting the input image into a threedimensional image of a threedimensional using the initial depth map generated by the depth map initialization unit 220 Referring to Figure 3 will be described.
Relative to the observer spider closest situated in the observer, then there is located a flower, the background will be described as an example in the farthest image from the observer, the image can be a spider, flowers, background separation in three steps according to a position Do. Of course more details from a possible, but will be divided into three stages as an example for description.
It is possible in this manner to the visual characteristics of subjective human analysis, but, if it be analyzed automatically through image processing using the prerequisite one spider that is considered to be the front of the achromatic and the depth map that is farthest because the darkness It will be represented.
Figure 3a is a view showing a threedimensional image transformation to the depth map in accordance with the preconditions 1.
Further, it jundamyeon change the depth map generated by the depth map initialization unit 220, such that the film video object as reverse printing of the photo film can be formed of a normal depth map.
That is, the depth map of the image is the initial depth map of Figure 3a when reversed as print film can be converted to depth map as shown in FIG 3b, and Figure 3b is capable of less than threedimensional representation error Figure 3a.
Figure 3b is an image of spider so the spider to the front and brightly expressed, followed by expression is dark flower region and background is due to the degree of change is substantially not. As a result, when selecting a depth map of Figure 3b, while improving the error can be expressed in a normal threedimensional.
Accordingly, the depth map generation apparatus 200 may select the image in the depth map of Fig. 3a to select whether or depth map of Figure 3b to be converted to determine conversion.
The frequency image and the input image performs the FFT (Fast Fourier Transform) conversion, to determine whether the depth map generation unit 200 is converted by selecting the depth map of Fig. 3a selected by converting the depth map whether or Figure 3b of produced, and by using the image frequency will determine the final depth map.
Accordingly, the depth map generation apparatus 200 includes a FFT conversion unit 230, determines the depth map 240.
The FFT conversion unit 230 converts the input image by performing an FFT on the input image to the image frequency in the frequency domain. That is, the FFT conversion unit 230 converts the input image on the spatial frequency in the image.
It will be the FFT conversion unit 230, a reference to Figure 4 for one to perform a FFT converts the input image into a video image frequency. 4, the more light portion has a vivid or strong outside. That is, the highfrequency region, i.e., the sharpness or when expressing part representing a strong outer shell to the light color to be outside of the spider brightly represented in the image and then the flower region and background is expressed darker because of the degree of change is substantially not in the frequency domain do.
(A) of Fig image expression such as 4 is to the nature of the FFT, Fig. 5 (b) of Fig would image an arbitrary region of 8x8 pixel in the input image 5 is the image of (a) into a frequency domain through a FFT.
A region in (b) of Figure 5 is to say the low frequency band, region B refers to a highfrequency band. In other words, the region B is a high frequency component, when the image is sharper or has a strong outside becomes large, the value of the area B, the more simple the value of the upper region A larger value of the area B becomes relatively small.
The sum of the pixel values belonging to the high frequency area in the frequency image, as, that of the total of the area B representing the sharpness in the 8x8 blocks (Block) expressing the variation between each pixel of the image large belonging to the partial area of the image screen 8x8 It is defined as a representative value corresponding to the pixel. To a size value of each of the block units as the brightness value is the one image representation utilizing FFT highfrequency component as shown in Fig. 4 can be represented as images in this way.
The depth map determination section 240 obtains a correlation value by using the average value of the representative value of the frequency image generated by the FFT conversion unit 230 and the initial depth map, and the final depth map on the basis of the correlation values to be determined.
That is, the depth map determination section 240 calculates a representative value (FFT (n)) representing the sharpness of any block of 8x8 block units in the frequency image formed through the FFT conversion unit 230. Here, the representative value is obtained summing pixel values corresponding to the high frequency area in the frequency image. Then determine the depth map 240, after obtaining the average value (Depth (n)) of the 8x8 block of the initial depth map matching block region of the FFT (n) by using the central value and the average value correlation calculate the value (CRV (CoRelation value)).
The depth map determination section 240 by using the equation (1) calculate the correlation value (CRV (CoRelation Value)).
Equation 1
CRV = Σ (FFT (n) * Depth (n))
Here, the FFT (n) is a representative value representing the block definition of the frequency image, and the Depth (n) is the average value of the initial depth map corresponding to a block area that matches the block area of the FFT (n), n is It means an index for each block (block). The FFT (n) refers to the sum of the pixel values corresponding to the high frequency region in the FFTconverted frequency image, the average value of the initial depth map from the initial depth map corresponding to a block area that matches the area obtained for the representative value, It means the average of the depth values.
The mathematical When the correlation value obtained by the equation 1, the depth map determination section 240 when the correlation value is not less than a predetermined threshold, and determining the initial depth map to a final depth map, other than the threshold value or more by inverting the depth value of the initial depth map determines that the inverted depth map to a final depth map.
As a result, by utilizing the frequency image conversion by the image in the frequency domain, such as the depth map determination section 240 is 4, the image automatically more appropriate for the depth map of Fig. 3b, rather than Fig. 3a to the depth map for converting the threedimensional the It will draw that conclusion.
Also, if the depth map determining unit 240 matches the condition of the condition it is determined that in front of the more the color of the preconditions 1 strong and bright colors and sharp parts at the front, that is, when the correlation value is equal to or greater than a predetermined threshold, a prerequisite it is determined that the depth map generation of FIG. 3a as the first sense.
If it is determined that the correlation value in this case is not equal to or greater than the predetermined threshold value, a depth map The inversion phenomenon is reflected as the depth map determination section 240 Figure 3b more reasonable depth map.
Depth map generation unit 200 depth map extraction an initial depth map by using the attribute information such as color, brightness information of the video object, and through the objective verification of this initial depth map partial or whether entirely reasonable constructed as described above to detect errors and generate a final depth map.
6 is a view showing how the threedimensional video conversion device according to the present invention converts an input image a twodimensional threedimensional image of a threedimensional.
6, the threedimensional image transformation apparatus extracts the at least one characteristic information by analyzing the input image of the twodimensional (S602). Here, the attribute information includes a boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, a histogram (histogram) information, and the like.
After execution of the S602, the threedimensional video conversion device generates an initial depth map for the input image based on the attribute information (S604).
Then the stereoscopic image converter determines the final depth map through the objective verification of said initial depth map whether partial or valid (S606).
That is, the threedimensional image converting unit converts the frequency image by performing FFT on the input image. Then, the threedimensional video conversion device calculates the representative value of the combined pixel values belonging to the high frequency area in the frequency image, it calculates the average value of the depth values corresponding to a block area that matches the area obtained for the representative value from the initial depth map. Then, the threedimensional video conversion device calculates the correlation value using the equation (1) using the representative values and the mean values the obtained.
Then, the stereoscopic image converter determines the final depth map on the basis of the correlation value.
After execution of the S606 the threedimensional image converting unit converts the input image using the final depth map with the determined threedimensional image of a threedimensional (S608).
7 is a flow chart illustrating a method of generating a depth map generation unit depth map in accordance with the present invention.
7, the depth map generation apparatus extracts the at least one property information for the input image, and generate the initial depth map for the input image based on the extracted characteristic information (S702). That is, the depth map generation apparatus extracts the characteristic information, such as the boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, a histogram (histogram) information. Then, the depth map generation apparatus may initially set the at least one block and then separated by a (block) the initial depth value for the at least one block (initial depth) of a plurality of pixels (pixel) forming the input image depth to generate a map (depth map).
After execution of the S702, the depth map generation apparatus may be transformed to the frequency image by performing FFT on the input image (S704).
After execution of the S704, the depth map generation unit may determine whether or not the image to the image frequency of the converted satisfy the precondition (S706). Here, the precondition is a two prerequisites, a prerequisite 2 "typically many, if the bright portion in the image is close to."
If the image that is determined in the S706 satisfy the precondition, the depth map generation apparatus may calculate the correlation value using the average value of the representative value and the initial depth map for the image frequency (S708). A detailed description of how to obtain the correlation value is illustrated in FIG.
After execution of the S708, the depth map generation unit may determine whether or not the calculated correlation value is greater than or equal to a predetermined threshold (S710).
If more than the threshold value determined result of the S710, the depth map generation apparatus may determine the initial depth map to a final depth map (S712).
If it is determined in the S710 the correlation value is not more than the threshold value, the depth map generation apparatus final depth map a depth map consisting of the inverting the depth value of the initial depth map and (S714), in the inverted depth values determines a (S716).
A computer program (also known as programs, software, software application, script, or code) for executing the method according to the present are included in the depth map generation apparatus or the threedimensional video conversion device according to the present invention invention is a compiled or interpreted language or a priori or procedures, and can be written in any form of programming language, including the language, can also be deployed in any form, including other unit suitable for use in standalone program or a module, component, subroutine, or the computer environment. A computer program does not necessarily have to correspond to the files in the file system. The program part of the inside in a single file that is provided to the requesting program, or the multiple crossfile serving (e. G., One or more modules, sub programs, or file for storing part of the code), or a file that holds other programs or data may be stored in the (e. g., one or more scripts stored in a markup language document). A computer program can be deployed so as to be located at one site or distributed across multiple sites running on a computer of a crossmultipleaccess computer or one by a communication network.
Medium readable by a suitable computer, for storing computer program instructions and data, for example, EPROM, EEPROM, and flash semiconductor memory device, such as a memory device, such as magnetic disks, such as internal hard disks and external disks, magnetooptical disks and CDROM and it shall include the DVDROM disc, including all forms of non volatile memory, media and memory devices. The processor and the memory is supplemented by a logic circuit or a specialpurpose, can be integrated into it.
Implementation of the functional operation and the topics discussed in this specification be implemented in digital electronic circuitry, computer software, including the structure and structural equivalents disclosed herein, implemented in firmware or hardware, or implemented with one or more combinations of these It can be. Implementation may be implemented within one or more computer program products of the subject described herein, that is to say one or more modules of the computer program instructions encoded on the type of program storage medium to or for the execution of this controlling the operation of a data processing device as it may be implemented.
Implementation of the subject matter described herein includes, for example, a backend component, such as a data server, or, for example, includes a middleware component, such as an application server, or, for example, your Web browser can interact implementation and mutual subject matter described in this specification or the graphical user It may be implemented in frontend component, or such backend, computing system comprising at least one combination of all of the middleware or a frontend component, such as a client computer having an interface. Of system components, for example it can be interconnected by any form or medium of digital data communication such as a communication network.
The specification includes a number of specific implementations of the detail, but these should not be any understood as inventions and also limit the scope of what can be charged, but rather a description of the features that can be specific to particular embodiments of the particular invention it should be understood. The specific features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. On the other hand, it is possible in a variety of technical features in the context of a single embodiment also implemented separately from or in Fig plurality of embodiments in any suitable subcombination. Further, characterized in that, but can be described as charge, such as that the operation and initially with particular combinations, one or more features from a claimed combination they can be excluded from the combination in some cases, that the claimed combination is subcombination or it may be modified by modification of the subcombination.
Similarly, although depicted in the drawings in a particular order of operation, which handageona be carried out such operations as the specific order or sequential order shown to achieve the desired result and should not be understood to be carried out by all illustrated acts. In certain cases, it may be advantageous are multitasking and parallel processing. In addition, the separation of various system components in the embodiments described above are not to be understood such a separation to be required in all embodiments, the program components and systems described above are generally to be integrated or packaged into multiple software products together in a single software product it can be understood that.
Thus, persons skilled in the art will appreciate that the present invention without changing the technical spirit or essential features may be embodied in other specific forms. Therefore, the embodiment described in the above examples should be understood as illustrative and not be limiting in all aspects. The scope of the invention is intended to be included within the scope of the above description becomes than indicated by the claims, which will be described later, the spirit and scope, and all such modifications as derived from the equivalent concept of the appended claims the invention do.
The present invention can be objectively detecting and correcting the error of the entire depth map of the image through the image processing for automatic threedimensional image transformation, by the transformation using the corrected depth map of the twodimensional image into a threedimensional video image It can be applied to a threedimensional image conversion device and method using the same and the depth map generation apparatus and method capable of minimizing the error in the conversion.
Claims (15)
 Characteristic for extracting at least one property information for the input image information extracting unit;Depth map initialization unit configured to generate an initial depth map for the input image based on the characteristic information;FFT conversion section that performs FFT (Fast Fourier Transform) to transform into the frequency image relative to the input image; AndUsing the average value of the representative value and the initial depth map for the image frequency to obtain a correlation value, the unit determines the depth map to determine the final depth map on the basis of the correlation value;Depth map generation apparatus including a.
 According to claim 1,Depth, characterized in that to extract the characteristic information including the characteristic data extraction unit boundary (edge) information, color (color) information, the brightness (luminance) information, the motion (motion) information, histogram at least one of (histogram) information map generating device.
 According to claim 1,The depth map initialization unit and then separate the pixels (pixel) a plurality of constituting the input image into at least one block (block) the initial depth of the initial depth map by setting (initial depth) value for the at least one block of ( depth map generation apparatus, characterized in that for generating a depth map).
 According to claim 1,After the depth map determination section calculated the average of the depth values corresponding to a block area that matches the area obtained for the representative value to obtain a representative value of the combined pixel values corresponding to the high frequency area in the frequency image, in the initial depth map, depth map generation apparatus, characterized in that to obtain the correlation values using the representative values and the mean value.
 The method of claim 4, whereinDepth map generation apparatus, characterized in that by using the equation of the depth map to the determination unit to obtain the correlation value (CRV (CoRelation Value)).Formula;CRV = Σ (FFT (n) * Depth (n))Here, the FFT (n) is the average value of the initial depth map corresponding to a block area that matches the block area of a single representative, the Depth (n) is FFT (n) representing the block definition of the frequency image, n are each It means an index of a block (block).
 According to claim 1,If there is more than the depth map determination section wherein the correlation value is a predetermined threshold value, when determining the initial depth map to a final depth map, and is not the threshold value or more by inverting the depth value of the initial depth map for the inverted depth map depth map generation apparatus, characterized in that to determine the final depth map.
 Analyzing the input image of the twodimensional image analysis unit to extract at least one attribute information;And then based on the attribute information creating an initial depth map for the input image, and converted into a frequency image by performing FFT on the input image, by using the average value of the representative value and the initial depth map of the frequency image obtaining a correlation value, setting the depth map unit for determining a final depth map on the basis of the correlation value; AndThreedimensional image generating unit for using the final depth map converting the input image into a threedimensional image of the threedimensional;Threedimensional video conversion device comprising a.
 The method of claim 7,The depth map setting section,Depth map initialization unit configured to generate an initial depth map for the input image based on the characteristic information;FFT converting section for performing the FFT to transform the image frequency relative to the input image; AndThreedimensional video conversion device comprising a representative value of the frequency image and the using the mean value of the initial depth map to obtain the correlation value, the portion determines the depth map to determine the final depth map on the basis of the correlation value.
 A method for the depth map generation apparatus generating a depth map,Extracting at least one feature information about the input image;Generating an initial depth map for the input image based on the characteristic information;Performing the FFT to transform the image frequency relative to the input image;Obtaining a correlation value by using the average value of the representative value and the initial depth map of said image frequency; AndDetermining a final depth map on the basis of the correlation value;How to create a depth map that includes.
 10. The method of claim 9,Obtaining a correlation value by using the average value of the representative value and the initial depth map for the image frequency, theObtaining a representative value of the combined pixel values corresponding to the high frequency area in the frequency image;The initial step in the depth map to obtain the average of the depth values corresponding to a block area that matches the area obtained by the representative value; AndHow to generate a depth map comprising the step of obtaining the representative value and the calculated correlation value using the average value.
 10. The method of claim 9,Determining a final depth map on the basis of the correlation values,Determining if the correlation value is not less than a predetermined threshold, if the initial depth map, not determined by the final depth map, and the threshold value or more by inverting the depth value of the initial depth map for the inverted depth map to a final depth map how to create a depth map features.
 Extracting at least one feature information about the input image;Generating an initial depth map for the input image based on the characteristic information;Performing the FFT to transform the image frequency relative to the input image;Obtaining a correlation value by using the average value of the representative value and the initial depth map of said image frequency;The correlation between the depth map generation determining a final depth map based on the value being recorded in the program recording medium readable in the electronic device.
 In the method the threedimensional video conversion device is to automatically convert the threedimensional image, theBy analyzing the inputted twodimensional image input step of extracting at least one attribute information;Further comprising: based on the attribute information creating an initial depth map for the input image, and determines the final depth map by checking whether the validity of the original depth map; AndThe method comprising using the final depth map converting the input image into a threedimensional image of the threedimensional;Threedimensional image conversion method comprising a.
 14. The method of claim 13,Determining the final depth map,Generating an initial depth map for the input image based on the characteristic information;Performing the FFT to transform the image frequency relative to the input image;Obtaining a representative value of the combined pixel values corresponding to the high frequency area in the frequency image;The initial step in the depth map to obtain the average of the depth values corresponding to a block area that matches the area obtained by the representative value;Obtaining a correlation value by using the average value of the representative value and the initial depth map of said image frequency; AndDetermining if the correlation value is not less than a predetermined threshold, if the initial depth map, not determined by the final depth map, and the threshold value or more by inverting the depth value of the initial depth map for the inverted depth map to a final depth map threedimensional image conversion method characterized in that comprises the step of.
 By analyzing the inputted twodimensional image input step of extracting at least one attribute information;Further comprising: based on the attribute information creating an initial depth map for the input image, and determines the final depth map by checking whether the validity of the original depth map; AndThe final depth map using a threedimensional image conversion method comprising the step of converting the input image into a stereoscopic image of the 3D program is recorded in a recording medium readable in the electronic device.
Priority Applications (2)
Application Number  Priority Date  Filing Date  Title 

KR1020110123374  20111124  
KR1020110123374A KR101660808B1 (en)  20111124  20111124  Apparatus and Method for generating Depth Map, stereoscopic image conversion apparatus and method usig that 
Publications (1)
Publication Number  Publication Date 

WO2013077508A1 true WO2013077508A1 (en)  20130530 
Family
ID=48469924
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

PCT/KR2012/003347 WO2013077508A1 (en)  20111124  20120430  Device and method for depth map generation and device and method using same for 3d image conversion 
Country Status (2)
Country  Link 

KR (1)  KR101660808B1 (en) 
WO (1)  WO2013077508A1 (en) 
Cited By (1)
Publication number  Priority date  Publication date  Assignee  Title 

KR20160095515A (en) *  20150203  20160811  연세대학교 산학협력단  3 dimensional optical measurement of defect distribution 
Citations (3)
Publication number  Priority date  Publication date  Assignee  Title 

KR20100019804A (en) *  20080811  20100219  한국전자통신연구원  Stereo vision system and control method thereof 
KR20100062360A (en) *  20081202  20100610  삼성전자주식회사  Device and method for depth estimation 
KR20100064196A (en) *  20081204  20100614  삼성전자주식회사  Method and appratus for estimating depth, and method and apparatus for converting 2d video to 3d video 
Family Cites Families (2)
Publication number  Priority date  Publication date  Assignee  Title 

KR100873638B1 (en) *  20070116  20081212  삼성전자주식회사  Image processing method and apparatus 
KR20100008677A (en)  20080716  20100126  광주과학기술원  Device and method for estimating death map, method for making intermediate view and encoding multiview using the same 

2011
 20111124 KR KR1020110123374A patent/KR101660808B1/en active IP Right Grant

2012
 20120430 WO PCT/KR2012/003347 patent/WO2013077508A1/en active Application Filing
Patent Citations (3)
Publication number  Priority date  Publication date  Assignee  Title 

KR20100019804A (en) *  20080811  20100219  한국전자통신연구원  Stereo vision system and control method thereof 
KR20100062360A (en) *  20081202  20100610  삼성전자주식회사  Device and method for depth estimation 
KR20100064196A (en) *  20081204  20100614  삼성전자주식회사  Method and appratus for estimating depth, and method and apparatus for converting 2d video to 3d video 
Cited By (2)
Publication number  Priority date  Publication date  Assignee  Title 

KR20160095515A (en) *  20150203  20160811  연세대학교 산학협력단  3 dimensional optical measurement of defect distribution 
KR101700109B1 (en) *  20150203  20170213  연세대학교 산학협력단  3 dimensional optical measurement of defect distribution 
Also Published As
Publication number  Publication date 

KR101660808B1 (en)  20160929 
KR20130057586A (en)  20130603 
Similar Documents
Publication  Publication Date  Title 

EP2157803B1 (en)  System and method for combining text with threedimensional content  
US8890934B2 (en)  Stereoscopic image aligning apparatus, stereoscopic image aligning method, and program of the same  
KR101483660B1 (en)  Method and apparatus for depthrelated information propagation  
US20180240265A1 (en)  Systems and Methods for DepthAssisted Perspective Distortion Correction  
JP5312676B2 (en)  Stereo camera with automatic control of interpupillary distance  
US8385595B2 (en)  Motion detection method, apparatus and system  
US20110043540A1 (en)  System and method for region classification of 2d images for 2dto3d conversion  
US20110298898A1 (en)  Three dimensional image generating system and method accomodating multiview imaging  
EP2153669B1 (en)  Method, apparatus and system for processing depthrelated information  
US8553972B2 (en)  Apparatus, method and computerreadable medium generating depth map  
US20110293170A1 (en)  Image processing apparatus and mathod  
WO2011014419A1 (en)  Methods, systems, and computerreadable storage media for creating threedimensional (3d) images of a scene  
JP2014502818A (en)  Primary image and the secondary image the image capture device and method for image processing  
CN1669337A (en)  Method for producing stereoscopic images from monoscopic images  
US8836760B2 (en)  Image reproducing apparatus, image capturing apparatus, and control method therefor  
CN102300112A (en)  The method of controlling a display apparatus and a display device  
CN102124490A (en)  Methods and systems for reducing or eliminating perceived ghosting in displayed stereoscopic images  
CN101437170B (en)  System and method for generating multieye visual image  
CN102741879A (en)  Method for generating depth maps from monocular images and systems using the same  
WO2012071356A1 (en)  Depth estimation based on global motion  
WO2010074437A2 (en)  Image processing method and apparatus therefor  
US20120113219A1 (en)  Image conversion apparatus and display apparatus and methods using the same  
US8963998B2 (en)  Full reference system for predicting subjective quality of threedimensional video  
US8208048B2 (en)  Method for high dynamic range imaging  
CN102215423B (en)  A method and apparatus for measuring parameters of audiovisual 
Legal Events
Date  Code  Title  Description 

121  Ep: the epo has been informed by wipo that ep was designated in this application 
Ref document number: 12852318 Country of ref document: EP Kind code of ref document: A1 

NENP  Nonentry into the national phase in: 
Ref country code: DE 

122  Ep: pct application nonentry in european phase 
Ref document number: 12852318 Country of ref document: EP Kind code of ref document: A1 