KR101169400B1 - A Method ＆ A System For Composing Stereo Image, And A Storage Medium - Google Patents

Abstract

The present invention relates to a stereo image forming method and system, and to a recording medium. The stereo image forming method according to the present invention comprises the steps of: decomposing a two-dimensional image into color segment regions; A feature point extraction step of detecting a feature line corresponding to an edge line by detecting an edge line from each of the color segment areas and referring to an object-background database storing feature point information of a target object; Dividing an area separated by the edge line into an object upon detection of a corresponding object feature point; A depth determination step of determining a two-dimensional depth value by detecting a size calculation and a background composition of the object from the object-background database when the area divided by the edge line is an object; And shifting the depth mask of the background-object formed by applying the two-dimensional depth value and performing image warping to form a stereo image.

Description

Method and system for forming stereo image and recording medium {A Method & A System For Composing Stereo Image, And A Storage Medium}

The present invention relates to a method and system for forming a stereo image, and more particularly, to detect object feature points of a two-dimensional image using data of an object-background database to distinguish a background from an object, and to determine the size, arrangement state, and position information of the object. The present invention relates to a stereo image forming method and system capable of defining a real depth of a two-dimensional image by converting the image into a left and right eye image and warping the image to enable live stereoscopic expression.

In the conventional stereo image forming method using two-dimensional images, the image is divided into a myriad of color segments, and the edge line is detected by considering the same object within the error range of the color for each color segment, and corresponding to the edge line. The present invention relates to a method of forming a stereoscopic stereo image by performing image warping based on depth information obtained by detecting a 2D image composition.

However, the conventional stereo image forming method has the following problems.

Detecting the edge line and detecting the corresponding two-dimensional image composition determines the depth information based on the relative position on the composition. Therefore, if there is an object with a figure similar to the composition, the composition detection itself becomes inaccurate. Since confusion in forming a three-dimensional effect, a problem arises in that the user must intuitively specify the depth value again.

The depth is limited to the background composition information obtained by the color segment, and the depth information is not included in the depth information because the information on the relationship between the main components of the image, which is the main information of the stereoscopic expression inherent in the 2D image, is not included. There has been a limit to the three-dimensional expression.

An object of the present invention is to detect the edge line from the color segments of the two-dimensional image, to distinguish the object and the background using the feature point information of the object corresponding to the edge line, and to detect the object composition by the size arrangement of the object In addition, the present invention is to implement a stereo image forming method and system for forming a stereo image by calculating a reliable depth value considering the correlation between the object composition and the background composition.

A stereo image forming method according to the present invention for achieving the above object comprises the steps of: decomposing a two-dimensional image into color segment regions; An edge line detection step of detecting an edge line from each of the color segment areas; An object storing feature point information on a target object; an object to extract a feature point corresponding to the detected edge line with reference to a background database; and to classify a region separated by the edge line into objects when a corresponding object feature point is detected; A background classification step; A depth determining step of determining a two-dimensional depth value by performing a size relationship calculation and a background composition detection on the object from the object-background database when the area divided by the edge line is an object; And shifting the depth mask of the background-object formed by applying the 2D depth value and image warping to form a stereo image.

In the depth determining step, an object composition vector is formed by detecting the size relationship and the position of the object, and when the zero point of the object composition vector is matched with the background composition reference vector, the magnitude value of the depth vector connecting the ends of the two vectors is determined. It is characterized in that to determine the depth value.

The object-background database may include: an object feature point database for storing target object feature point data for distinguishing whether an area divided by the edge line is an object or a background; It is characterized in that it comprises a composition database (Composition Database) including the figure data for detecting the composition of the background.

The object-background database is characterized in that it comprises a pseudo-database (Diagram Database) for storing pseudo-shape data for the accuracy of the background composition or object size information detection.

The data of the object feature point database and the composition database are stored as binary data to reduce the amount of computation by object feature point detection and composition detection.

The object feature point database includes vector processing data for feature points of an object so that the object feature point database can be applied to extraction of objects of various sizes.

The depth determining step may further include a step of a user designating a background composition or an object size of a two-dimensional image, which is difficult to automatically set by the information of the object-background database, to an arbitrary value.

A stereo image forming system according to the present invention for achieving the above object comprises: segment forming means for decomposing a two-dimensional image into color segment regions; Edge line detection means for detecting an edge line from each of said color segment areas; An object-background database that stores data for object information detection and background composition detection; Object-background discrimination means for extracting a feature point corresponding to the detected edge line from the object-background database and discriminating whether an area divided by an edge line corresponding to the feature point is an object or a background; Depth determining means for detecting a size relationship and a background composition of the object from the object-background database to determine a two-dimensional depth value when the area divided by the edge line is an object; And a stereo image forming means for shifting the depth mask of the background-object formed by applying the two-dimensional depth value and performing image warping to form a stereo image.

A computer-readable recording medium according to the present invention for achieving the above object is characterized in that a computer program for implementing a stereo image forming method on a computer system is recorded.

According to the above configuration, it is possible to generate a live stereo image by calculating reliable depth information based on the size relationship of the object shown in the two-dimensional image and the characteristics of the background composition, and the object feature point database and the background composition Reduce the amount of computation by storing data such as databases as binary information.

1 is a block diagram illustrating a stereo image generation process according to the present invention.
2 is a diagram illustrating an edge line detection state by segmentation of a two-dimensional image of the present embodiment.
3 is a diagram illustrating an edge line detection state by segmentation of a two-dimensional image of the present embodiment.
4 is a diagram illustrating a configuration of an object feature point database in an embodiment of the present invention.
5 is a diagram illustrating a state where a feature point is detected from an edge line in the embodiment of the present invention.
FIG. 6 illustrates a pseudographic that includes all of the feature points of each object. FIG.
7A is a diagram illustrating an original 2D image to be converted according to an embodiment of the present invention.
FIG. 7B illustrates a state in which color segments are formed on the original image of FIG. 7A. FIG.
FIG. 7C illustrates object composition vectors corresponding to objects 1 and 2 of FIG. 7B.
FIG. 7B illustrates a background composition vector corresponding to the background of FIG. 7B. FIG.
FIG. 7C is a diagram illustrating an object composition vector and a background composition vector for the object and the background of FIG. 7B.
8 is a diagram illustrating a configuration of a composition database for detecting a background composition.
9A and 9B are diagrams illustrating a depth value setting state by user setting.
10A is a diagram illustrating a shift direction of a depth mask of a left eye image and a right eye image for a determined arbitrary depth value.
10A illustrates a state prior to shifting an image depth mask.
FIG. 10C illustrates a state after shifting the image depth mask according to the depth value determined for the depth mask of FIG. 10A.

Hereinafter, a configuration of a preferred embodiment of a stereo image forming method and system according to the present invention having the configuration as described above will be described in detail with reference to the accompanying drawings.

The stereo image forming system of the illustrated embodiment of the present invention includes segment forming means for decomposing a two-dimensional image into color segment regions, edge line detection means for detecting edge lines from the segment regions, and various objects; An object-background database storing data about a background and a feature point corresponding to the detected edge line are extracted to detect a target object in the object-background database, and to detect whether an area divided by the edge line is an object or a background. Depth-determination means for determining a two-dimensional depth from object-background detection means, object object information (size relationship, arrangement, etc.) corresponding to object feature points, and composition information of the background, and the two-dimensional depth; Image warpins for objects and background masks formed by applying decision values and g) performing a stereo image forming means to form a stereo image.

In this embodiment, as shown in Fig. 2, the two-dimensional image is divided into countless cells and one color is determined in each cell before edge detection, thereby forming color segments. For example, the formation of the segment is not limited to the color segment (step 1 and step 2).

The color segmentation process as described above makes it possible to clearly distinguish the area of the object because the color change (gradation) gradually made in the existing image is divided into clear colors as shown in FIG. 2.

That is, in the case of a clear image, objects can be distinguished without forming a segment, but in a case of a non-clear image, since it is difficult to directly distinguish an object area, a color segment can be formed to clearly distinguish the object.

As shown in FIG. 2, the edge line detection unit detects an edge surrounding an area within an error range of a color difference from a color segment of a two-dimensional image.

For example, as shown in FIG. 3, the white clothing part of the child has a color difference between the face, the arm, or the background and the color segment at the boundary point, and the part corresponding to the white clothes is within the error range. An edge line surrounding the part is detected (third step).

The object-background separating means of the embodiment of the present invention removes and normalizes portions unnecessary for distinguishing the object-background or grasping the background composition from the edge lines detected by the edge line detecting means.

That is, in the state where the edge line is detected as shown in the upper right of FIG. 3, fine spots are visible but are unnecessary for object detection, and are removed by normalization to obtain clear edge lines. On the other hand, the process of forming the segment for the two-dimensional image prior to the edge detection can be removed to some extent by itself (third step).

In addition, the object-background discrimination means of the embodiment of the present invention is an object feature point detection algorithm for the edge line normalized by the above process (comparison and repetition of the object feature point data of the object feature point database described below and the area divided by the edge line). Feature point detection and target object detection by learning). For example, as shown in FIG. 4, each vertex of the cube may be detected as a feature point, and as shown in FIG. 5, various feature points corresponding to an edge line may be compared and detected from the object-background database, and the feature points of the database may also be detected. A target object corresponding to may be detected (fourth step).

The object-background database of an embodiment of the present invention is an object feature point database that stores data for distinguishing whether an area separated by an edge line is an object or a background, for example, data about object feature points formed at various angles of a target object. A point database, a composition database that stores figure data for detecting a background composition, and figure data for accuracy of detecting a background composition or detecting object object information (e.g., object size array, etc.) by object feature points. It includes a pseudogram database (Diagram Database) for storing.

The background-object detection means of this embodiment detects whether or not the area distinguished by the edge line is an object or a background by detecting feature points corresponding to the edge line with reference to the above-described object-background database. For example, an embodiment of the present invention detects whether a target object of a feature point corresponding to a normalized edge line exists in the object feature point database, and if it exists in the object feature point database, detects an area separated by an edge line as an object. In the case of detection in the background (step 5).

As shown in FIG. 4, the object feature point database of the present embodiment stores data about object feature points formed at various angles of the target object as binary data (black and white or 0 and 1), thereby reducing the amount of computation. The feature point data is processed in vector scale so that the detection can be performed accurately.

Meanwhile, in the embodiment of the present invention, when the area divided by the normalized edge line is detected as the background, object information about the background composition is detected from the data stored in the composition database and used to determine the object depth value.

For example, the composition database illustrated in FIG. 8 is for recognizing a composition of a background such as horizontal, vertical, diagonal, diagonal, curved, radial, triangle, inverted triangle, center point, symmetrical, tunnel, and checkerboard composition. Stores data needed for geometry- i.e. triangles, squares, rectangles, circles, rhombuses.

In addition, in the embodiment of the present invention, the data corresponding to the area divided by the edge line is extracted from the composition database of FIG. 8 to determine the composition of the two-dimensional image. In this case, the background composition may be configured by a combination of one or two specific figures, and when the background composition is determined, the depth of the 2D image may be primarily determined by the relative position on the composition.

In the embodiment of the present invention, the pseudo-shape database is used to obtain information about the size or depth of the object only by the background composition or the object feature point, and is used to obtain information about the object, and to correct errors in the composition or object detection. Used as

For example, the pseudo-graphics database stores data about a pseudo-figure, which is a figure that includes all object feature point regions as a figure (eg, square, circle, rectangle, equilateral triangle, etc.) approximating an edge line, and in this embodiment, Pseudo-figures that include all the feature points of an object are used for accurate detection of the object's size, arrangement, composition, and position.

For example, as illustrated in FIG. 6, the size of each object may be determined by the size of a similarity shape (rectangle in FIG. 6) including all object feature points, and the relative size data of each object may be compared to determine the size of the object. Relative position and depth values can be obtained.

That is, the relative size of the object and the spatial position of the object by the relative size of the pseudo-figure (for example, two small rectangles approximating each of the two cylindrical barrels of Fig. 9) that includes all the feature points of the tool storage box that is the object having the same feature point structure We can get the depth value for.

The composition database and the pseudo-graphic database of this embodiment store the information as binary data (for example, the data value is expressed in black or white or 0 or 1) as the object feature point database, and thus the background composition and the depth of the object-background Reduce the amount of computation for value determination.

Depth determination means according to an embodiment of the present invention is the automatic setting means for automatically calculating the depth value by the data of the object feature point database, the composition database and the pseudo-figure database, and the depth value by the case of difficult automatic determination or selection of the operator Or determining the depth value of the two-dimensional image, including user setting means for manually specifying the composition (sixth, seventh, and eighth steps).

The process of determining the depth by the automatic setting means of the present embodiment is as follows. That is, the object composition vector is detected using the size information of the object of each position detected in the 2D image (the diagonal lines of FIGS. 7C and 7E), and the background composition reference vector is detected by detecting the background composition.

For example, since the size of object 2 is small when comparing object 1 and object 2 on the image, the object composition direction may be set in the direction of object 1 to object 2 and the object composition vector may be detected.

On the two-dimensional image as shown in Figs. 7A to 7E, the background composition is detected as the horizontal composition, and a reference vector of the background composition in the form of a vertical line corresponding to the horizontal composition is obtained (see the vertical vector in Fig. 7D).

The process of detecting the depth vector using the reference vector of the background composition and the object composition vector is as follows. That is, the depth value of the object is automatically set by matching the zero point of the object composition vector and the reference vector of the background composition and constructing a depth vector which is a vector connecting the ends of the two vectors (step 7).

Meanwhile, in the case of user setting, as shown in FIG. 9A, a depth value is determined by directly designating an object composition vector with respect to a reference vector of the two-dimensional image background composition. In the present embodiment, the object composition vector designation may be performed by directly designating the size of the object.

For example, as shown in FIG. 9B, the first vector is a reference vector of the two-dimensional image depth, and when the second vector is an object composition vector, the depth of the object is three times connecting the first and second vector ends. Can be defined as a vector (step 6).

In the present invention, since the depth value of the two-dimensional image is compensated according to the object size and relative position in addition to the background composition, vivid stereoscopic images can be formed. In addition, if the setting by the automatic setting means is difficult or the state of the image to which the depth value is applied is poor, such as when the depth value due to the size of the object is out of the error range by the automatic setting means, The composition information or the depth value may be directly input (sixth step).

In addition, when directly inputting or automatically setting the composition information or the depth information, the operation may be repeatedly performed several times until an appropriate final value for the three-dimensional representation is determined.

The stereo image forming means of the present embodiment forms a stereo image for the left and right eyes by shifting and warping the object to a position suitable for the representation of the three-dimensional effect according to the depth value determined by the depth determining means.

for example. As shown in Fig. 10A, the stereo image forming means of this embodiment forms a depth mask of an object-background according to a given depth value and shifts the mask from side to side according to the depth value of the object based on the reference value. To form a left eye image and a right eye image.

Referring to the case of the left eye image in the present embodiment, when the depth value of the object is small based on the reference value, that is, when the image should be seen, the depth mask of the object is shifted to the left, and when the object is to protrude, The depth mask is shifted to the right to form a left eye image (ninth step).

Meanwhile, referring to the right eye image in this embodiment, when the depth value of the object is small based on the reference value, that is, when the image should be seen, the depth mask of the object should be shifted to the right side (center) and protruded. In this case, the depth mask of the object is shifted to the left (center) to form a right eye image.

Currently, all three-dimensional display devices (TV, monitor, mobile) commercially available have different sizes of monitors, and three-dimensional monitors are manufactured by different technologies for different hardware manufacturers, so the reference value of yin and yang of depth expression is different. For this reason, the image mask must be moved to a step that matches the depth mask value to the hardware. That is, the depth mask value is divided by each step and moved by 1 pixel per step to implement an appropriate depth for each hardware.

For example, when the depth reference value is 110, a smaller than 110 shifts a pixel of the 2D image to the left to a left to form a left eye image. Also, smaller than 110 shifts to the right and larger than 110 to the left to form the right eye image. On the other hand, the number of steps to shift the depth mask can be determined according to the characteristics of the hardware, the value is determined by the maximum protruding depth value, the maximum visible depth value and the depth limit values according to the hardware characteristics .

Meanwhile, as shown in FIG. 10B and FIG. 10C, the empty space (see FIG. 10C) generated by the depth mask shift in the left and right directions around the depth reference value is filled by image warping to complete the left and right eye images. (The ninth step).

The scope of the present invention is not limited to the above embodiments, but is determined by the matters described in the claims, and it is obvious that the present invention includes various modifications and adaptations made by those skilled in the art in the claims and equivalents.

Claims (9)

Decomposing the two-dimensional image into color segment regions;
An edge line detection step of detecting an edge line from each of the color segment areas;
An object storing feature point information on a target object; an object to extract a feature point corresponding to the detected edge line with reference to a background database; and to classify a region separated by the edge line into objects when a corresponding object feature point is detected; A background classification step;
A depth determining step of determining a two-dimensional depth value by performing a size relationship calculation and a background composition detection on the object from the object-background database when the area divided by the edge line is an object;
Shifting the depth mask of the background-object formed by applying the two-dimensional depth value and image warping to form a stereo image.
The method of claim 1, wherein in the depth determination step,
Detecting the size relationship and the position of the object to form an object composition vector, and if the zero of the object composition vector with respect to the background composition reference vector to match the zero value of the depth vector connecting the ends of the two vectors to determine the depth value Stereo image forming method, characterized in that.
The method of claim 1, wherein the object-background database,
An object feature point database for storing target object feature point data for distinguishing whether an area divided by the edge line is an object or a background;
And a composition database including composition data for detecting a composition of a background.
The method of claim 1, wherein the object-background database,
And a pseudogram database storing pseudo figure data for accurate background composition or object size information detection.
The method according to claim 3, wherein the data of the object feature point database and the composition database are stored as binary data to reduce the amount of computation by object feature point detection and composition detection.
The method of claim 3, wherein the object feature point database includes vector processing data of feature points of an object so that the object feature point database can be applied to extraction of objects of various sizes.
The method of claim 1, wherein the depth determination step,
And a user designating a background composition or an object size of the two-dimensional image, which is difficult to set automatically by the information of the object-background database, to an arbitrary value.
Segment forming means for decomposing the two-dimensional image into color segment regions;
Edge line detection means for detecting an edge line from each of said color segment areas;
An object-background database that stores data for object information detection and background composition detection;
Object-background discrimination means for extracting a feature point corresponding to the detected edge line from the object-background database and discriminating whether an area divided by an edge line corresponding to the feature point is an object or a background;
Depth determining means for detecting a size relationship and a background composition of the object from the object-background database to determine a two-dimensional depth value when the area divided by the edge line is an object;
And stereo image forming means for shifting the depth mask of the background-object formed by applying the two-dimensional depth value and performing image warping to form a stereo image.
A computer-readable recording medium having recorded thereon a computer program for implementing the stereo image forming method according to any one of claims 1 to 7 on a computer system.

Images