KR101323195B1 - 3D image generating method from 2D images - Google Patents

Abstract

The present invention relates to a method for generating a 3D image from a 2D image, and more particularly, to a 3D display such as a 3D TV or a 3D screen from a 2D image photographed using a plurality of general cameras without using a separate stereo photographing apparatus. The present invention relates to a method of generating a 3D image from a 2D image for generating a 3D image that can be viewed in a stereoscopic manner through a device.
The present invention for achieving the above object is an image capture and input step of storing the first and second images taken by the first and second cameras in the image storage unit provided in the image conversion apparatus, and in the first and second images A first, a transform relationship setting step of setting a reference frame for selecting a reference frame, a conversion relationship between the reference frames determined in the reference frame determination step, and a conversion relationship set in the conversion relationship setting step are used. And converting the two images to generate a left image for the left eye and a right image for the right eye.

Description

3D image generating method from 2D images

The present invention relates to a method for generating a 3D image from a 2D image, and more particularly, to a 3D display such as a 3D TV or a 3D screen from a 2D image photographed using a plurality of general cameras without using a separate stereo photographing apparatus. The present invention relates to a method of generating a 3D image from a 2D image for generating a 3D image that can be viewed in a stereoscopic manner through a device.

Generally, 3D image generation methods include a method of directly photographing a 3D image using a stereo camera and a method of converting a 2D image captured by a general camera into a 3D image.

Among these methods, 3D video recording using a stereo camera uses a special shooting device called a stereo rig. A left eye camera and a right eye camera are installed at a specific position of the stereo rig, and both cameras simultaneously capture the same subject. 3D video is generated by shooting.

These stereo rigs have a fixed range of fixation of the left and right cameras so that the fixed range and the left and right cameras can be rotated to have a wider shooting range. Both types of stereo rigs require the user to manually align the left and right cameras when the camera is mounted, and manually adjust the zero-point parallax, which is the center of the three-dimensional experience. The disadvantage is that the alignment and zero parallax adjustment process must be repeated.

In the existing 3D image generating method, a method of converting a single 2D image photographed by a general camera into a 3D image is input manually by inputting distance information between a subject and a camera photographed in a single 2D image and using the 3D image. This is how you create it.

As an example of a method of generating such a 3D image, there is a technology described in Korean Patent No. 10-0793076 as shown in FIGS. 1 and 2. The technical feature of the method is an edge-adaptive stereo / multi-view image matching device. Edge extracting means (102) and edge extracting means (102) for extracting an edge from an image to be matched from the image acquiring means; In the variation search range using the matching window frame selecting means 103 for selecting the matching window frame to be applied according to the edge distribution type and the matching window frame selected by the matching window frame selecting means 103 by examining the distribution of the edges extracted in 102). A variation search means 104 for searching for a variation in the variation map storage means 106 for storing the variation as a variation map. The matching window frame selecting means 103 is configured to examine the distribution of the edges extracted by the edge extraction means 102 and apply one of a plurality of matching window frames that are previously classified according to the distribution form of each edge component. Characterized in determining the matching window frame.

By the way, the technology described in Korean Patent Registration No. 10-0793076 has the advantage of generating a 2D image as a 3D image, but in order to capture the 2D image must be provided with a camera installed at a specific position specific angle, for each camera There is a problem that a user must directly input the information, and a disparity map is necessary to generate a 3D image.

The present invention has been made to solve the above problems, an object of the present invention is to automatically from two or more 2D images taken with a general camera even if the camera is not a 2D image taken by mounting the camera at a specific position specific angle, such as a stereo rig The present invention provides a method of generating a 3D image from a 2D image capable of generating a 3D image.

Another object of the present invention is to easily generate a 3D image from a 2D image by directly calculating a relative position and angle even without inputting information such as an angle or a distance of a camera for two or more 2D images captured by a general camera. It is to provide a method for generating a 3D image from a 2D image.

SUMMARY OF THE INVENTION [0006]

Image capture and input step of storing the first and second images taken by the first and second cameras in the image storage unit provided in the image conversion device, and the step of determining the reference frame to select the reference frame from the first and second images, respectively And a conversion relationship setting step of setting a conversion relationship between the reference frames determined in the reference frame determination step, and converting the first and second images by using the conversion relationship set in the conversion relationship setting step, and left image for the left eye. And a 3D image generation step of generating a right image for the right eye and the right eye.

The converting relationship setting step may include: extracting a corresponding point corresponding to each other in a reference frame of the first and second images, and using the corresponding point extracted in the corresponding point extracting process. Set the relative position and angle calculation process for calculating the relative position and the relative angle, and the conversion relationship for generating the left image for the left eye and the right image for the right eye in consideration of the relative position and the relative angle of the first and second cameras. It is characterized by consisting of a conversion relationship setting process.

In the process of calculating the relative position and angle, the relative position and the relative angle of the first and second cameras are calculated using a relative facial expression element measuring technique.

Further, in the step of determining the reference frame, if the photographing scene is changed or the focal length of the camera is changed from each of the first image and the second image, the frame corresponding to the case where the camera position and the photographing direction is changed is extracted and extracted respectively. The frame of the second image and the first image photographed at the same time as the frame of the first image and the second image is selected and set as a reference frame that can be used for establishing an image transformation relationship.

According to the present invention having the above-described configuration, even if the 2D image taken by mounting the camera at a specific position specific angle, such as a stereo rig, there is an effect that can automatically generate a 3D image from two or more 2D images taken with a general camera .

In addition, the present invention is an effect that can easily generate a 3D image from the 2D image by directly calculating the relative position and angle even without inputting information such as the angle or distance of the camera for two or more 2D images taken with a general camera There is.

1 is a block diagram of a conventional 3D image registration device.
2 is a flowchart of a conventional 3D image registration method.
3 is a block diagram of a 3D image generating apparatus in a 2D image according to the present invention.
4 is a flowchart of a method for generating a 3D image from a 2D image according to the present invention.
5 is a flowchart illustrating a detailed process of establishing a conversion relationship of a 3D image generating method in a 2D image according to the present invention.
6 is a conceptual diagram illustrating a relationship between corresponding points of a first image and a second image according to the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention. The same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted. It is to be understood that the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

3 is a block diagram of an apparatus for generating a 3D image from a 2D image according to the present invention, FIG. 4 is a flowchart of a method of generating a 3D image from a 2D image according to the present invention, and FIG. 5 is a 3D image generated from a 2D image according to the present invention. 6 is a flowchart illustrating a detailed process of establishing a conversion relationship of a method, and FIG. 6 is a conceptual diagram illustrating a relationship between corresponding points of a first image and a second image according to the present invention.

The present invention relates to a method for generating a 3D image from a 2D image by using an image captured by a general camera. First, an image converting apparatus 200 for generating a 3D image will be described with reference to FIG. 220, the reference frame extractor 280, the correspondence point detector 280, the relative position calculator 230, the conversion relationship storage 240, the image converter 250, and the main processor for controlling the above-described components. It consists of 210.

Here, the image storage unit 220 stores the first and second images taken by the first camera 260 and the second camera 270, and the reference frame extractor 280 stores the first and second images. When the shooting scene is changed or the focal length of the camera is changed from, the reference frame corresponding to the case where the position and the shooting direction of the camera is changed is extracted, the correspondence point detector 290 corresponding points corresponding to each other in the reference frame The relative position calculating unit 230 calculates the relative position and angle of the first camera 260 and the second camera 270 from the corresponding points.

The conversion relationship storage unit 240 converts the first and second images into a left image for the left eye and a right image for the right eye in consideration of the relative positions and angles of the first camera 260 and the second camera 270. The conversion relationship for conversion is set and saved.

In addition, the image conversion unit 250 converts the first and second images using the conversion relation stored in the conversion relationship storage unit 240 to generate a left image and a right image that can feel a three-dimensional effect.

In addition, referring to the method of generating a 3D image from the 2D image using the image conversion apparatus 200, as illustrated in FIG. 4, the image conversion apparatus may capture the first and second images captured by the first and second cameras 260 and 270. Image capture and input step (S100) and the reference frame determination step (S200) and the reference frame determination step of selecting the reference frame from the first and second images, respectively, stored in the image storage unit 220 provided in (200) The left and right for the left eye by converting the first and second images using the conversion relationship setting step S300 for setting the conversion relationship between the reference frames determined in S200 and the conversion relationship set in the conversion relationship setting step S300. 3D image generation step (S400) for generating the image and the right image for the right eye.

Here, in the step of determining the reference frame (S200), when the photographing scene is changed or the focal length of the camera is changed from each of the first and second images through the reference frame extractor 280, the position and the photographing direction of the camera are changed. The frame corresponding to the changed case can be extracted, and the frames of the second image and the first image photographed at the same time as the extracted first image and the second image frame can be selected and used as pairs to establish the relationship between the images. It is set as the reference frame.

And, as shown in Figure 5, the conversion relationship setting step (S300), in the correspondence point extraction process (S310) and the correspondence point extraction process for extracting a corresponding point corresponding to each other in the reference frame of the first and second images The left image for the left eye in consideration of the relative position and angle calculation process (S320) of calculating the relative position and the relative angle of the first and second cameras using the extracted corresponding points and the relative position and the relative angle of the first and second cameras. And a conversion relationship setting process of setting a conversion relationship for generating a right image for the right eye and the right eye.

Here, in the corresponding point extraction process (S310), a point representing the same part of the subject is extracted from the corresponding reference frame extracted from the first and second images and set as a corresponding point.

In this case, since the first and second images are not photographed at the same position and the photographing angles of the first and second cameras 260 and 270 are different from each other, the corresponding point is the reference of the first image as shown in FIG. 6. Corresponding points A, B and C extracted from a frame and corresponding points A ', B' and C 'extracted from a reference frame of the second image form offsets of different positions and angles when compared in the same frame. Done.

Thus, in the relative position and angle calculation process (S320) in consideration of the position of the corresponding points (A, A ', B, B', C, C '), using the relative facial expression element measurement technique, the first and second The relative position and relative angle between the cameras 250 and 260 are calculated.

)를 산출하게 된다.That is, the projection center of the first camera 260 (or the second camera 270) and the first camera 260 using the corresponding points A, A ', B, B', C, and C 'of the reference frame. Position of the second camera 270 (or the first camera 260) (or generally, three-dimensional coordinates X, Y, Z) and the second based on the photographing direction of the second camera 270 (or the second camera 270). 3-axis rotation angle of the camera's shooting direction (the normal three-dimensional angle

).

In this case, for example, the relative point coordinates of the first and second cameras 250 and 260 are calculated to correspond to the corresponding point coordinates of the reference frame of the first image (x, y) and the corresponding point coordinates of the reference frame of the second image. When (x ', y'), the focal length of the first camera 260 is f, the focal length of the second camera is f ', the relative position and relative angle between the first and second cameras 260, 270 is the first image It can be calculated using the following relationship between the corresponding point vector Sq of the reference frame of, the corresponding point vector S'q 'of the reference frame of the second image, and the relative position vector B of the first and second cameras.

Meanwhile, in the conversion relationship setting process (S330), Y parallax (up and down) generated due to the relative position and angle difference between the first and second cameras 260 and 270 using the position and angle obtained in the relative position and angle determination process (S320). Parallax) is removed and the first and second images are converted to match the scale between the first and second cameras 260 and 270 so as to set a conversion relationship for generating a left image and a right image.

Here, as an example of the conversion relationship setting process (S330), if the coordinate of the image point before the conversion is (x, y) and the coordinate of the image point after the conversion is (xx, yy), the coefficient values of the conversion relation are as follows. Can be extracted from relative angles.

In the 3D image generating step (S400), the left and right eyes for the left eye and the right eyes for the right eye are applied to all frames of the first and second images by applying the conversion relationship set in the conversion relationship setting step (S300). Will be converted.

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the scope of the present invention is not limited to the disclosed embodiments, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

The present invention relates to a method for generating a 3D image from a 2D image, and more particularly, to a 3D display such as a 3D TV or a 3D screen from a 2D image photographed using a plurality of general cameras without using a separate stereo photographing apparatus. The present invention relates to a method of generating a 3D image from a 2D image for generating a 3D image that can be viewed in a stereoscopic manner through a device.

200: video conversion device 210: main processor
220: image storage unit 230: relative position calculation unit
240: conversion relationship storage unit 250: image conversion unit
260: first camera 270: second camera
280: reference frame extraction unit 290: correspondence point detection unit
300: subject
L: Left eye R: Right eye

Claims (4)

An image capturing and input step of storing the first and second images captured by the first and second cameras in an image storage unit provided in the image converting apparatus;
A reference frame determining step of selecting reference frames from the first and second images, respectively;
A conversion relationship setting step of setting a conversion relationship between the reference frames determined in the reference frame determining step;
3D image generation step of generating the left image for the left eye and the right image for the right eye by converting the first and second images using the conversion relationship set in the conversion relationship setting step,
The conversion relationship setting step,
A correspondence point extraction process of extracting correspondence points corresponding to each other from the reference frames of the first and second images;
A relative position and angle calculation process of calculating a relative position and a relative angle of the first and second cameras by using a relative expression factor measuring technique using the corresponding points extracted in the corresponding point extraction process;
A conversion relationship setting process of setting a conversion relationship for generating a left image for the left eye and a right image for the right eye in consideration of the relative position and the relative angle of the first and second cameras,
In the step of determining the reference frame, when the shooting scene is changed or the focal length of the camera is changed from each of the first image and the second image, a frame corresponding to the case where the position and the shooting direction of the camera is changed is extracted and the first image is extracted. In the two-dimensional image, characterized in that the frame of the second image and the first image photographed at the same time as the frame of the second image and select the pair and set it as a reference frame that can be used to establish the relationship between the image conversion 3D image generation method.
delete delete delete

Images