KR20130001635A - Method and apparatus for generating depth map - Google Patents

Abstract

PURPOSE: A depth map producing method and an apparatus thereof are provided to express a three-dimensional effect which is more realistic than in a process converting from 2D image data to 3D data by producing a proper optimal depth map according to the analysis result of an image and preset conditions. CONSTITUTION: A pre-processing unit(110) performs the analysis and classification of input 2D image data. A conversion unit(120) selects a conversion algorithm corresponding to the classified image. The conversion unit produces a depth map according to the combination of a conversion algorithm. A post-processing unit(130) performs a depth filter compensation stabilizer about 3D image data. [Reference numerals] (A1,A2) Scene detection point; (BB) Motion-based tracking; (C1,C2) Motion field; (DD) MTD detection; (EE) Scene classification; (FF) Focus analysis; (GG) Context analysis; (HH) Saliency analysis; (II) Natural analysis; (JJ) Artificial analysis; (KK) Focus depth; (LL) Context depth; (MM) Object depth; (NN) Alpha depth; (OO) Vertical/edge; (PP) Depth filter compensation stabilizer

Description

Method and apparatus for generating depth map {Method And Apparatus for Generating Depth MAP}

One embodiment of the present invention relates to a method and apparatus for generating a depth map. More specifically, in the process of converting 2D image data into 3D image data, various types of images are analyzed and classified according to analysis results and preset conditions in order to generate a depth map for expressing a more realistic three-dimensional effect. The present invention relates to a depth map generation method and apparatus for providing 3D image data to which depth map information combining an optimal transformation algorithm for each classified image is applied.

The contents described in this section merely provide background information on the embodiment of the present invention and do not constitute the prior art.

As digital technology is highly developed and broadcasting media are diversified due to convergence of broadcasting and communication, broadcasting-related additional services using the characteristics of digital technology are newly introduced. Currently, the direction of the development of video services such as TV is moving toward high definition and large screens, but since it provides only 2D video contents, viewers cannot feel a three-dimensional effect through the current video contents.

Accordingly, the necessity of the stereoscopic image is gradually raised, but the content of the stereoscopic image is still insufficient. Stereoscopic image processing technology is a core technology in the field of next-generation information and communication services, and is a cutting-edge technology in which competition for technology development is fierce with the development of the information industry society. Such stereoscopic image processing technology is an essential element to provide high quality video services in multimedia applications. Today, the application fields such as broadcasting, medical, education, military, game, and virtual reality are diversified. have.

Accordingly, there is a need for a technology of providing 2D image content as stereoscopic image content. However, the current technology has a problem that it is difficult to express a precise three-dimensional effect. Of course, it is possible to provide a precise stereoscopic image conversion effect in a manual method, but when working in such a manual method, there is a problem that a person must perform a task for a very long time using visual cognitive ability.

In order to solve the above-described problem, an embodiment of the present invention provides a depth map for applying depth map information generated by using a combination of transformation algorithms according to characteristics of an image in a process of converting 2D image data into 3D image data. The main purpose is to provide a method and apparatus.

In order to achieve the above object, an embodiment of the present invention, a pre-processing unit for analyzing and classifying the input 2D image data; And a transform unit for selecting a transform algorithm corresponding to the classified image and generating a depth map according to a combination of the transform algorithms.

In addition, according to another object of the present invention, a pre-processing step of analyzing and classifying the input 2D image data; And a transformation step of selecting a transformation algorithm corresponding to the classified image and generating a depth map according to a combination of the transformation algorithms.

As described above, according to an embodiment of the present invention, in the process of converting 2D image data into 3D image data, various types of images are analyzed to generate a depth map for expressing a more realistic three-dimensional effect, and the analysis result and Images are classified according to preset conditions, and the depth map information combining the optimal conversion algorithms according to the classified images may be provided.

In addition, according to an embodiment of the present invention, in order to minimize an inappropriate error occurring when generating a depth map through a conventional single-image processing algorithm, an optimal depth map according to an analysis result of an image and a predetermined condition There is an effect that can generate. In addition, according to an embodiment of the present invention, it is possible to classify images according to the types of images and to increase the efficiency of the finally generated depth map by selecting an optimal algorithm suitable for the characteristics of each classified image and combining them. It has an effect. In addition, according to an embodiment of the present invention, to provide a process for classifying the image reflecting the visual characteristics in order to achieve the effect of human cognitive ability, and by applying an image processing algorithm that makes an optimal effect on such classification It is effective to express three-dimensional effect.

1 is a block diagram schematically illustrating an apparatus for generating a depth map according to an embodiment of the present invention;
2 is a table showing a first condition for image classification according to an embodiment of the present invention;
3 is a table showing a second condition for a combination of transform algorithms according to an embodiment of the present invention;
4 is a table illustrating a third condition for determining a conversion algorithm reliability according to an embodiment of the present invention;
5 is a flowchart illustrating a depth map generation method according to an embodiment of the present invention;
6 is an exemplary diagram for generating a depth map according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings an embodiment according to the present invention will be described in detail.

The first condition in which the present invention is described is information including a condition for classifying a 2D image, and includes at least one or more of a modified time difference (MTD) condition, a scene detection condition, and an image characterization condition. The second condition may include combination information according to a combination of image feature information, and refers to information in which a plurality of algorithms and a predetermined depth map ratio match each combination information, and the third condition corresponds to an algorithm included in the second condition. Refers to information including probability information.

1 is a block diagram schematically illustrating an apparatus for generating a depth map according to an exemplary embodiment of the present invention.

The depth map generator 100 according to the present invention refers to an apparatus for automatically converting 2D image data into 3D image data. In this case, the depth map generator 100 converts 2D image data into 3D image data. In the following description, an object or a background is divided and separated from 2D image data, and an appropriate depth map is assigned to each object to give a sense of distance. Finally, 3D image data corresponding to the left view image and the right view image are generated through a rendering operation. This operation may be performed manually by the user, but may be performed through a conversion algorithm as in the present invention.

On the other hand, to describe the 3D image data, the representation of the 3D image data (stereoscopic image data) basically requires two images taken from the left and right, such as the human eye. That is, two images taken independently from the left and the right are respectively shown to the human eye, and the images taken from the left independently to the left eye, and the images taken from the right to the right eye to feel a three-dimensional feeling. Therefore, a broadcast camera for capturing a stereoscopic image has two cameras attached thereto or has two lens units. Since the stereoscopic image requires the left image and the right image, two images corresponding to the left and right sides should be created in the process of converting the 2D image data into the 3D image data.

The process of converting the 2D image data into the 3D image data by the depth map generating apparatus 100 may be performed by manual operation or an automated operation by a program. Of course, the automation task is a method using a conversion algorithm, but it is difficult to perform a task such as manual work or a realistic work process according to the human cognitive ability, but the result can be significantly improved through an appropriate conversion algorithm according to the image. For example, in general, a process of generating a depth map for generating a 3D image from a 2D image is a manual method in which a person directly separates an object and assigns a depth map (value) for each object. However, in the case of the automatic method, conventional image processing methods such as motion analysis, frequency analysis, and histogram analysis are used, which may cause an error because it is impossible to judge the human cognitive ability. In other words, separating objects (people or cars) from multiple backgrounds is instantaneous if a person works manually, while automatic programmatic separation of objects or cars from objects is generally possible. it's difficult. However, if appropriate conditions are given as in the present invention, this error can be minimized and image processing can be performed. That is, if there is a motion of a person or a car, it is possible to separate from the background without motion through motion information, or by using a blur phenomenon according to distance, a distinction between near and distant objects is normally provided on the premise that a clear object is near. It is possible. Or it is difficult to express precise features using structural features of the background, but it is also possible to create a depth map corresponding to the entire background. That is, the image is a variety of forms, but as described above, if the characteristic conditions of the image are analyzed, it is possible to apply the optimal conversion algorithm accordingly.

The depth map generating apparatus 100 includes a preprocessor 110, a converter 120, and a postprocessor 130 to convert 2D image data into 3D image data. Of course, in one embodiment of the present invention, the depth map generating apparatus 100 is described as including only the preprocessor 110, the converter 120 and the post-processor 130, which is described in the embodiment of the present invention As merely illustrative of the idea, those skilled in the art to which an embodiment of the present invention belongs may be included in the depth map generating apparatus 100 without departing from the essential characteristics of the embodiment of the present invention. Various modifications and variations to the components will be applicable.

Referring to the general operation of the preprocessor 110, the preprocessor 110 performs an operation for classifying images. Since the 2D image data is composed of a set of consecutive frames, the 2D image for each frame having similar characteristics as possible. Image classification is performed to group the data so that the same transformation algorithm can be applied.

Referring to the main operations performed by the preprocessor 110, the preprocessor 110 analyzes and classifies the input 2D image data. In this case, the preprocessor 110 uses a preset first condition to classify the 2D image data. Here, to briefly describe the first condition, the first condition includes at least one or more of an MTD condition, a scene detection condition, and an image feature providing condition.

The MTD condition, the scene detection condition, and the image characteristic provision condition included in the first condition will be described. The MTD condition is a condition for selecting a frame corresponding to the MTD from the 2D image data, and the scene detection condition is a MTD detection from the 2D image data. The scene change point and the frames within the scene change point are grouped into similar scene groups for the remaining image data not classified as images, and the image characterization condition is analyzed by analyzing similar scene groups for each group to perform the image characterization condition. It is a condition to give image characteristic information. For example, an image grouped into a similar scene group includes a current frame f (n), a next frame f (n + 1), a next frame f (n + 2), and the like.

In addition, the process of performing the MTD condition by the preprocessor 110 will be described in detail. In order to perform the MTD detection condition, the preprocessor 110 analyzes the 2D image data and horizontally in the 2D image data. A frame f (m) in which horizontal panning of a moving object or a camera occurs is classified as an MTD and grouped into an MTD detection image. In this case, the preprocessor 110 may perform at least one of motion vector analysis, horizontal camera panning, and horizontal moving to analyze the 2D image data when performing the MTD condition. Use technology

In addition, the process of performing the scene detection condition by the preprocessor 110 will be described in detail. In order to perform the scene detection condition, the preprocessor 110 performs the rest of the image not classified as the MTD detection image of the 2D image data. For the data, scene change points and frames within the scene change points are grouped into similar scene groups. At this time, the preprocessor 110 sets the current frame f (n) among the remaining image data not classified as MTD among the 2D image data as a scene detection point, and then histogram information based on the current frame. For example, the frame until the point at which one of the edge information and the motion information is inconsistent over a preset threshold is recognized as a scene change point. At this time, the preprocessing unit 110 compares the current frame and the next frame to determine a portion to be changed when switching from the current frame to the next frame, and if the determined variation is not correlated between the current frame and the next frame, It is determined as one similar scene group.

In addition, the process of the preprocessing unit 110 to perform the image characteristic provision condition will be described in detail. The preprocessing unit 110 analyzes similar scene groups for each group to perform the image characteristic provision condition. Grant. The image characteristic information may include at least one of static, dynamic, close, non-close, natural, and artificial images. Include. At this time, the preprocessing unit 110 uses at least one or more of a motion vector, a global motion vector, a local motion vector, edge change information, and histogram information to perform an image characterization condition. The dynamics are checked and classified into either static or dynamic image based on the confirmed result. In addition, the preprocessing unit 110 may perform a similar scene using at least one or more information of spectrum analysis, blur analysis, feature detection, and focus analysis to perform image characterization conditions. Groups are classified into either near or far images. In addition, the preprocessor 110 performs spectral analysis, edge change information, color histogram information, alpha blending algorithm, and horizontal edge emphasis to perform image characterization conditions. Similar scene groups are classified into either natural image or artificial image using at least one or more information of (Horizontal Edge Emphasis).

Referring to the general operation of the transform unit 120, the optimal conversion algorithm corresponding to the classified image by the preprocessor 110 is selected and applied to the 2D image data. Meanwhile, the conversion unit 120 performs object extraction / segmentation, depth map computation, and rendering / occclusion in the process of converting 2D image data into 3D image data. .

Meanwhile, to describe the main operation process of the converter 120 according to an embodiment of the present invention, the converter 120 selects a conversion algorithm corresponding to the image classified by the preprocessor 110. Next, a depth map according to the combination of the conversion algorithms is generated. In this case, the conversion unit 120 uses the second condition to select a conversion algorithm for the classified image that has performed the image characterization condition through the preprocessor 110. Here, the second condition has combination information according to the combination of the image feature information, and is information in which a plurality of algorithms and a predetermined depth map ratio are matched for each combination information. In addition, the plurality of algorithms included in the second condition may include an alpha blending algorithm, a context depth algorithm, a vertical depth algorithm, an object depth algorithm, and a focus depth. At least one of the algorithms. In addition, the depth map ratio included in the second condition may correspond to at least one of a main depth, a sub 1 depth, a sub 2 depth, and an optional depth. Contains the depth map ratio.

In detail, a process of generating the depth map by the transform unit 120, the transform unit 120 selects a transform algorithm corresponding to the image classified by the preprocessor 110, and selects the transform algorithm. Create a depth map according to the combination. Meanwhile, the transform unit 120 may generate a depth map in a state where the reliability of the transform algorithm is not reflected. To explain this, the transform unit 120 is included in the classified image that performs the image characterization condition. The combination information matching the second condition is extracted according to the image feature information, and the reliability non-reflective depth map ratio is calculated based on the conversion algorithm matched with the combination information and the depth map ratio. At this time, the conversion unit 120 calculates the reliability unreflected depth map by using Equation 1.

(A, B, C: each algorithm, α: preset main depth ratio, β: preset sub 1 depth ratio, γ: preset sub 2 depth ratio)

Meanwhile, the transformation unit 120 may generate a depth map in a state where the reliability of the transformation algorithm is reflected. To this end, the transformation unit 120 may be configured to determine a transformation algorithm fidelity. Use conditions. Here, the third condition includes probability information according to an algorithm included in the second condition. The converter 120 calculates the reliability reflection depth map by using Equation 2.

(A, B, C: each algorithm, α`: probability information of preset main depth ratio XA, β`: probability information of preset sub-1 depth ratio × B, γ`: preset sub-2 depth ratio × C Probability information)

Meanwhile, the converter 120 generates 3D image data by applying a depth map to 2D data. That is, the process of generating the 3D image data in which the converter 120 applies the depth map to the 2D data will be described in detail. The converter 120 may include a left view image and a right view image to which the depth map is applied. Generates rendered 3D image data. In addition, the converter 120 generates an image for the left view or an image for the right view, which is 3D image data, using the MTD detection image grouped by the preprocessor 110.

Referring to the general operation of the post-processing unit 130, the 3D image data is stabilized in order to increase the completeness of the 3D image data. In other words, the operation performed by the post-processing unit 130 will be described in more detail. The post-processing unit 130 performs a depth filter compensation stabilization on 3D image data to reflect the depth map reflected in the 3D image data. If the information does not exceed the preset threshold, it is confirmed that the 3D image data is stabilized.

2 is a table showing a first condition for image classification according to an embodiment of the present invention.

The first condition illustrated in FIG. 2 includes at least one or more of an MTD condition, a scene detection condition, and an image characteristic condition. That is, the first condition illustrated in FIG. 2 represents a condition for classifying 2D image data in order to select the most optimal conversion algorithm according to the feature or condition of the image. That is, three types of standards for classifying 2D image data are classified as illustrated in FIG. 2.

First, the MTD condition of item 1 of FIG. 2 is a condition for selecting a frame corresponding to the MTD from the 2D image data. That is, to describe the MTD condition, the depth map generating apparatus 100 analyzes 2D image data and performs horizontal panning of an object or a camera moving horizontally in the 2D image data in order to perform the MTD condition (f). (m)) is classified as MTD and grouped by MTD detection image. In this case, the depth map generating apparatus 100 uses at least one of motion vector analysis, horizontal camera panning, and horizontal object movement to analyze 2D image data when performing the MTD condition. In addition, the depth map generating apparatus 100 groups the scene change point and the frames in the scene change point into similar scene groups with respect to the remaining image data not classified as the MTD detection image among the 2D image data in order to perform the scene detection condition.

Item 2 of FIG. 2 is a scene detection condition, and the scene detection condition is a condition for grouping scene transition points and frames in the scene change points into similar scene groups for the remaining image data not classified as MTD detection images among 2D image data. That is, to describe the scene detection condition, the depth map generating apparatus 100 sets the current frame f (n) among the remaining image data not classified as MTD among the 2D image data as the scene detection time point, and then the current frame. Based on this, a frame until a point at which one of the histogram information, the edge information, and the motion information is inconsistent over a predetermined threshold value is recognized as a scene change point. In this case, the depth map generating apparatus 100 compares the current frame with the next frame to determine a portion that is changed when the current frame is switched from the current frame to the next frame, and when the determined variation is not correlated between the current frame and the next frame. In other words, it is determined as an independent similar group of scenes.

Item 3 of FIG. 2 is a condition for granting image feature information by analyzing similar scene groups for each group to perform the image feature grant condition. That is, to describe the image feature providing condition, the depth map generating apparatus 100 analyzes a similar scene group for each group and assigns the image feature information to perform the image feature providing condition. Here, the image feature information includes at least one or more of a static image, a dynamic image, a near image, a far image, a natural image, and an artificial image. At this time, the depth map generating apparatus 100 uses at least one of motion vector, global motion vector, local motion vector, edge change information, and histogram information to perform an image characterization condition. The motion or dynamics are checked and classified into either static or dynamic image based on the result of the confirmation. In addition, the depth map generating apparatus 100 may perform at least one of a near scene image and a far image having a similar scene group using at least one or more information of spectrum analysis, blur analysis, feature point detection, and focus analysis to perform an image characteristic condition. Classify. In addition, the depth map generating apparatus 100 may perform a similar scene group using at least one of spectrum analysis, edge change information, color histogram information, alpha blending algorithm, and horizontal edge emphasis to perform image characterization conditions. Classify into either image or artificial image.

3 is a table showing a second condition for a combination of transform algorithms according to an embodiment of the present invention.

When the classification of the input 2D image data is completed based on the first condition, an optimal algorithm determined to be most valid may be selected based on this. In this case, the depth map generating apparatus 100 may use the second condition illustrated in FIG. 3. That is, when the appropriate algorithms are arranged in order for each of the images classified based on the first condition, they are the same as the second condition shown in FIG. 3.

The second condition has combination information according to a combination of image feature information, and is information in which a plurality of algorithms and a predetermined depth map ratio are matched for each combination information. The plurality of algorithms included in the second condition include at least one or more of an alpha blending algorithm, a context depth algorithm, a vertical depth algorithm, an object depth algorithm, and a focus depth algorithm. Also, the depth map ratio included in the second condition includes a depth map ratio with respect to at least one of a main depth, a sub 1 depth, a sub 2 depth, and an optional depth.

When the image characteristic information corresponding to the classified image is checked, a combination of specific algorithms is determined as in the second condition of FIG. 3, and the corresponding depth map specific gravity is a main depth, a sub 1 depth, a sub 2 depth, and an optional depth. Divided into In addition, as in the second condition of FIG. 3, the ratio of the depth map may have an initial ratio (initial representative value) of 60%, 25%, 10%, and 5%. Such initial ratios are merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may vary the initial ratio in various ways without departing from the essential characteristics of the present invention. It will be applicable to the modification. In addition, the depth map reflecting the initial ratio may be determined as shown in [Equation 1] according to the characteristics of the image as an automated scheme in the process of image processing. In the case of including the reliability, the depth map may be determined according to the image characteristics. 2]. That is, the depth map generating apparatus 100 extracts the combination information corresponding to the second condition according to the image feature information included in the classified image that performs the image feature providing condition, and converts the depth algorithm and the conversion algorithm matched to the combination information. Based on the ratio, the reliability unreflected depth map ratio is calculated. The depth map generating apparatus 100 calculates the reliability unreflected depth map by using Equation 1.

As an example of the second condition of FIG. 3, when the image feature information of the classified image is checked, in the case of the image classified as the static image, the near image, and the natural image, the depth map 'S main depth is' Object Depth', and each sub-depth 1 and 2 in the background is in the order of 'alpha blending depth' and 'focus depth' in the ratio of 60%, 25% and 10%, respectively. Is determined. Therefore, as a result, the depth map is set to the value summed up according to the ratio of each depth.

4 is a table showing a third condition for determining a conversion algorithm reliability according to an embodiment of the present invention.

The combination of the selected conversion algorithms is determined according to the first condition and the second condition, and a depth map of the result summed at an appropriate ratio thereof can be obtained, but it is necessary to additionally reflect the reliability of each algorithm. The reason for reflecting the algorithm reliability is the uncertainty of the images classified through the first condition. In other words, the image processing may include an error, and even if the classification of the image itself is confirmed by the human eye, there is uncertainty even when automatically determining the classification of the image. Therefore, the probability information is reflected in the results classified to compensate for these uncertainties.

For example, when the distinction between the near and far images is difficult to determine, such as ON or OFF, it is expressed as a probability through image processing, and the spectral analysis is performed on the corresponding images. The support vector machine (SVM) determines which direction the image is classified. At this time, in this process, the ratio is determined through learning, and accordingly, a numerical element of which to classify the corresponding image comes out and this corresponds to a relative probability to be classified as a result.

Therefore, as in the third condition of FIG. 4, each reliability is set for each corresponding algorithm. At this time, the probability is not determined but the result value is reflected for each image classification process. On the other hand, if the probability is more than 50%, it may be a selection criterion for classifying the algorithm. In this case, the criteria for classification reflect additional means such as the third condition because they cannot be considered absolutely complete even if the probability exceeds 50%.

That is, the depth map generator 100 may use Equation 2 to generate a depth map reflecting such reliability. Here, the depth map generating apparatus 100 uses a preset third condition to determine the conversion algorithm reliability. The depth map generating apparatus 100 calculates the reliability reflection depth map by using Equation 2. As a result, in the process of combining the algorithms selected according to the second condition of the depth map generating apparatus 100 and reflecting the respective depth map ratios, the probability information is reflected in the third condition and the depth map is more reliable based on the probability information. Can be calculated.

5 is a flowchart illustrating a depth map generation method according to an embodiment of the present invention.

The main operation performed by the depth map generating apparatus 100 will be described. The depth map generating apparatus 100 performs a preset first condition in order to analyze and classify the input 2D image data S510 (S520). . Here, to briefly describe the first condition, the first condition includes at least one or more of an MTD condition, a scene detection condition, and an image feature providing condition. That is, the MTD condition, the scene detection condition, and the image feature granting condition included in the first condition will be described. The MTD condition is a condition for selecting a frame corresponding to the MTD from the 2D image data, and the scene detection condition is selected from the 2D image data. The scene change point and the frames within the scene change point are grouped into similar scene groups with respect to the remaining image data not classified as the MTD detection image, and the image characterization condition is a similar scene group for each group to perform the image characterization condition. It is a condition that gives image characteristic information by analyzing.

In detail, a process of performing the MTD condition by the depth map generating apparatus 100 in step S520 will be described in detail. In order to perform the MTD condition, the depth map generating apparatus 100 analyzes 2D image data and performs horizontal processing on the 2D image data. A frame f (m) in which horizontal panning of an object moving to or a camera occurs is classified as an MTD, and grouped into an MTD detected image. In this case, the depth map generating apparatus 100 uses at least one of motion vector analysis, horizontal camera panning, and horizontal object movement to analyze 2D image data when performing the MTD condition.

In addition, the depth map generating apparatus 100 performs a scene detection condition (S530). A process of performing the depth map generating apparatus 100 by S530 may be performed by grouping scene change points and frames in the scene change points into similar scene groups with respect to the remaining image data which is not classified as the MTD detection image among the 2D image data. In addition, the process of performing the scene detection condition by the depth map generating apparatus 100 will be described in more detail. The depth map generating apparatus 100 may include a current frame among the remaining image data not classified as MTD among 2D image data. After setting f (n)) as the scene detection time, a frame up to a time point at which one of the histogram information, edge information, and motion information is inconsistent over a preset threshold is recognized as a scene change point. In this case, the depth map generating apparatus 100 compares the current frame with the next frame to determine a portion that is changed when the current frame is switched from the current frame to the next frame, and when the determined variation is not correlated between the current frame and the next frame. In other words, it is determined as an independent similar group of scenes.

The depth map generating apparatus 100 performs an image feature providing condition (S540). In operation S540, the depth map generating apparatus 100 performs an image feature providing condition. The depth map generating device 100 analyzes a similar scene group for each group to perform an image feature providing condition. Give feature information. Here, the image feature information includes at least one or more of a static image, a dynamic image, a near image, a far image, a natural image, and an artificial image. At this time, the depth map generating apparatus 100 uses at least one of motion vector, global motion vector, local motion vector, edge change information, and histogram information to perform an image characterization condition. The motion or dynamics are checked and classified into either static or dynamic image based on the result of the confirmation. In addition, the depth map generating apparatus 100 may perform at least one of a near scene image and a far image having a similar scene group using at least one or more information of spectrum analysis, blur analysis, feature point detection, and focus analysis to perform an image characteristic condition. Classify. In addition, the depth map generating apparatus 100 may perform a similar scene group using at least one of spectrum analysis, edge change information, color histogram information, alpha blending algorithm, and horizontal edge emphasis to perform image characterization conditions. Classify into either image or artificial image.

That is, the depth map generating apparatus 100 performs an operation for classifying an image through steps S510 to S540. Since the 2D image data is composed of a set of continuous frames, the depth map generating apparatus 100 performs 2D image data for each frame having similar characteristics as possible. The image classification process is performed by grouping to apply the same conversion algorithm.

The depth map generating apparatus 100 selects a conversion algorithm corresponding to the classified image on which the image characteristic condition is performed (S550). When performing step S550, the depth map generating apparatus 100 uses the second condition to select a conversion algorithm for the classified image that has performed the image characteristic condition. Here, the second condition has combination information according to the combination of the image feature information, and is information in which a plurality of algorithms and a predetermined depth map ratio are matched for each combination information. Also, the plurality of algorithms included in the second condition may include at least one of an alpha blending algorithm, a context depth algorithm, a vertical depth algorithm, an object depth algorithm, and a focus depth algorithm. Also, the depth map ratio included in the second condition includes a depth map ratio with respect to at least one of a main depth, a sub 1 depth, a sub 2 depth, and an optional depth.

The depth map generating apparatus 100 generates a depth map according to a combination of transformation algorithms (S560). In more detail with reference to step S560, the depth map generating apparatus 100 may generate the depth map in a state where the reliability of the conversion algorithm is not reflected. The combination information matching the second condition is extracted according to the image feature information included in the classified image that performs the feature assignment condition, and the reliability non-reflective depth map ratio is calculated based on the conversion algorithm matched with the combination information and the depth map ratio. do. At this time, the depth map generating apparatus 100 calculates the reliability unreflected depth map by using Equation 1. Meanwhile, the depth map generator 100 may generate a depth map in a state where the reliability of the transform algorithm is reflected. To this end, the depth map generator 100 may convert the depth algorithm to the depth map generator 100. Use the third predetermined condition to determine the. Here, the third condition includes probability information according to an algorithm included in the second condition. The depth map generating apparatus 100 calculates the reliability reflection depth map by using Equation 2.

The depth map generating apparatus 100 generates 3D image data to which the depth map is applied to the 2D data (S570). In more detail with reference to step S570, the depth map generating apparatus 100 generates 3D image data in which a left view image and a right view image to which the depth map is applied are rendered. In addition, the depth map generating apparatus 100 generates a left view image or a right view image, which is 3D image data, using the grouped MTD detection image. The depth map generating apparatus 100 stabilizes the 3D image data in order to increase the completeness of the 3D image data (S580). In more detail with reference to step S580, the depth map generating apparatus 100 performs depth filter compensation on the 3D image data so that the depth map information reflected in the 3D image data does not exceed a preset threshold, and the 3D image Confirm that the data is stabilized.

In FIG. 5, steps S510 to S580 are described as being sequentially executed. However, this is merely illustrative of the technical idea of an embodiment of the present invention, and the general knowledge in the technical field to which an embodiment of the present invention belongs. Those having a variety of modifications and variations may be applicable by changing the order described in FIG. 5 or executing one or more steps of steps S510 to S580 in parallel without departing from the essential characteristics of an embodiment of the present invention. 5 is not limited to the time series order.

As described above, the depth map generation method according to an embodiment of the present invention described in FIG. 5 may be implemented in a program and recorded in a computer-readable recording medium. A computer-readable recording medium having recorded thereon a program for implementing a method for generating a depth map according to an embodiment of the present invention includes all kinds of recording devices storing data that can be read by a computer system. Examples of such computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also implemented in the form of a carrier wave (e.g., transmission over the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code is stored and executed in a distributed manner. In addition, functional programs, code, and code segments for implementing an embodiment of the present invention may be easily inferred by programmers skilled in the art to which an embodiment of the present invention belongs.

6 is an exemplary diagram for generating a depth map according to an embodiment of the present invention.

When the image (A) shown in FIG. 6 is a static image, a close image, and classified as a natural image according to the first condition, as illustrated in FIG. According to the second condition, 100 extracts a 'bird' according to feature point detection as an object, and as shown in FIG. 6C, image characteristic information obtained from an original image in 'background' and 'object'. By using the 'alpha blending method' can be obtained the final result. Meanwhile, 'focus depth' may be additionally applied.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

As described above, the present invention is applied to a field for applying depth map information generated by using a combination of conversion algorithms according to characteristics of an image in a process of converting 2D image data into 3D image data, thereby converting 2D image data to 3D. In the process of converting to image data, various types of images are analyzed to classify images according to analysis results and preset conditions to generate a depth map for expressing a more realistic three-dimensional effect, and optimal conversion for each classified image is performed. It is a useful invention to generate an effect capable of providing depth map information combining algorithms.

100: depth map generator 110: preprocessor
120: analysis unit 130: post-processing unit

Claims (36)

A preprocessor for analyzing and classifying the input 2D image data; And
A transform unit that selects a transform algorithm corresponding to the classified image and generates a depth map according to a combination of the transform algorithms.
Depth map generating device comprising a. The method of claim 1,
The preprocessing unit,
And a predetermined first condition is used to classify the 2D image data. The method of claim 2,
The first condition is,
And at least one or more of a modified time difference (MTD) condition, a scene detection condition, and an image characterization condition. The method of claim 3, wherein
The MTD condition is,
And a condition for selecting a frame corresponding to MTD among the 2D image data. The method of claim 3, wherein
The preprocessing unit,
In order to perform the MTD condition, an object moving horizontally from the 2D image data and a frame f (m) where horizontal panning of the camera occurs from the 2D image data are generated. Depth map generation device characterized in that classified by the MTD, grouped by the MTD detection image. The method of claim 5, wherein
The preprocessing unit,
And at least one of motion vector analysis, horizontal camera panning, and horizontal moving to analyze the 2D image data. The method of claim 5, wherein
Wherein,
And a left view image or a right view image, which is 3D image data, using the MTD detection image. The method of claim 5, wherein
The preprocessing unit,
In order to perform the scene detection condition, a depth map generating apparatus for grouping scene change points and frames in scene change points into similar scene groups with respect to the remaining image data not classified as the MTD detection image among the 2D image data. . The method of claim 8,
The preprocessing unit,
Among the 2D image data, the current frame f (n) among the remaining image data not classified as the MTD is set as a scene detection point, and then histogram information, edge information, and motion based on the current frame. And a frame until a time point at which any one of the information does not match above a preset threshold as the scene change point. The method of claim 9,
The preprocessing unit,
Compares the current frame with a next frame to determine a portion to be changed when switching from the current frame to the next frame, and an independent one if the determined variation is not correlated between the current frame and the next frame And determine a similar scene group. The method of claim 8,
The preprocessing unit,
Depth map generation device characterized in that to analyze the similar scene group for each group to give the image feature information to perform the image feature assignment condition. The method of claim 11,
The video feature information,
At least one of static, dynamic, close, non-close, natural and artificial images. Depth Map Generator. 13. The method of claim 12,
The preprocessing unit,
In order to perform the image characterization condition, at least one or more of a motion vector, a global motion vector, a local motion vector, edge change information, and histogram information is used to identify a motion or dynamics of an object in the similar scene group. And a depth map generating apparatus classified into one of the static image and the dynamic image based on a result of the checking. 13. The method of claim 12,
The preprocessing unit,
In order to perform the image characterization condition, the similar scene group is associated with the near image using at least one or more information of spectrum analysis, blur analysis, feature detection, and focus analysis. And a depth map generator according to one of the distant images. 13. The method of claim 12,
The preprocessing unit,
In order to perform the image characterization condition, at least one of spectrum analysis, edge change information, color histogram information, alpha blending algorithm, and horizontal edge emphasis And the similar scene group is classified into one of the natural image and the artificial image by using one or more pieces of information. 13. The method of claim 12,
Wherein,
And a second condition is used to select the transform algorithm for the classified image that has performed the image characteristic condition. 17. The method of claim 16,
The second condition is that,
Depth map generation device having combination information according to the combination of the image feature information, the plurality of algorithms and a predetermined depth map ratio is matched for each combination information. The method of claim 17,
The plurality of algorithms,
And at least one of an alpha blending algorithm, a context depth algorithm, a vertical depth algorithm, an object depth algorithm, and a focus depth algorithm. Depth map generator. The method of claim 17,
The depth map ratio is,
A depth map comprising a depth map ratio for at least one of a Main Depth, a Sub 1 Depth, a Sub 2 Depth, and an Optional Depth Generating device. The method of claim 19,
Wherein,
Extracting the combination information corresponding to the second condition according to the image feature information included in the classified image that has performed the image feature providing condition, and extracts the combination information corresponding to the combination information and the depth map ratio. And a depth map ratio is calculated based on the reliability unreflected depth map ratio. 21. The method of claim 20,
Wherein,
Depth (f) = αA + βB + γC
(A, B, C: each algorithm, α: preset main depth ratio, β: preset sub 1 depth ratio, γ: preset sub 2 depth ratio)
Depth map generation device characterized in that for calculating the reliability unreflected depth map using the equation. The method of claim 18,
Wherein,
Depth map generating device using a third predetermined condition to determine the conversion algorithm (Fidelith). The method of claim 22,
The third condition is that,
And depth information generating apparatus according to the algorithm included in the second condition. 24. The method of claim 23,
Wherein,
Depth (f`) = α`A + β`B + γ`C
(A, B, C: each algorithm, α`: probability information of preset main depth ratio XA, β`: probability information of preset sub-1 depth ratio × B, γ`: preset sub-2 depth ratio × C Probability information)
Depth map generation device characterized in that for calculating the reliability reflected depth map using the equation. The method of claim 1,
Wherein,
And generating 3D image data to which the depth map is applied to the 2D data. The method of claim 25,
Wherein,
And generating the 3D image data in which the left view image and the right view image to which the depth map is applied are rendered. The method of claim 1,
And a post-processing unit for stabilizing the 3D image data. The method of claim 27,
The post-
When the depth map information reflected in the 3D image data does not exceed a preset threshold by performing depth filter compensation stabilization on the 3D image data, it is confirmed that the 3D image data is stabilized. Depth map generator. A preprocessing step of analyzing and classifying the input 2D image data; And
A transformation step of selecting a transformation algorithm corresponding to the classified image and generating a depth map according to a combination of the transformation algorithms
Depth map generation method comprising a. 30. The method of claim 29,
The pretreatment step,
And a first condition including at least one of a Modified Time Difference (MTD) condition, a Scene detection condition, and an image feature providing condition to classify the 2D image data. 31. The method of claim 30,
The preprocessor step,
In order to perform the MTD condition, an object moving horizontally from the 2D image data and a frame f (m) where horizontal panning of the camera occurs from the 2D image data are generated. Depth map generation method characterized in that classified by the MTD, grouped by the MTD detection image. 31. The method of claim 30,
The pretreatment step,
In order to perform the scene detection condition, a depth map generation method for grouping scene transition points and frames in scene transition points into similar scene groups for the remaining image data not classified as the MTD detection image among the 2D image data. . 31. The method of claim 30,
The pretreatment step,
And analyzing the similar scene group for each group and assigning image feature information to perform the image feature providing condition. 34. The method of claim 33,
The conversion step,
And a second condition is used to select the transform algorithm for the classified image which has performed the image characteristic condition. 35. The method of claim 34,
The conversion step,
Extracting the combination information corresponding to the second condition according to the image feature information included in the classified image that has performed the image feature providing condition, and extracts the combination information corresponding to the combination information and the depth map ratio. Calculating a depth map ratio without reflecting reliability based on the depth map generation method. The method of claim 18,
The conversion step,
And a third condition including probability information according to an algorithm included in the second condition to determine a transform algorithm reliability.

Images