KR20150091064A - Method and system for capturing a 3d image using single camera - Google Patents

Abstract

A method of capturing a 3D image using a single camera comprises capturing a first image as a right or left image in a first position by a single camera; Extracting feature points of the first image; Capturing a picture to find a second image as the other image in a position different from the first position; Extracting feature points of a picture; Comparing the feature points of the first image and the picture; Generating two 3D cursors, one of the cursors representing a target position of the second image and the other representing a current position of the camera; Displaying two 3D cursors on a picture; Capturing a second image when the cursor indicating the current position completely overlaps the cursor indicating the target position by moving and rotating the camera; And synthesizing the first and second images to produce a 3D image.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and system for capturing a 3D image using a single camera,

The present invention relates to a method and system for capturing 3D images using a single camera. More particularly, the present invention relates to a method and system for capturing 3D images using a single camera by using 3D cursors.

In the nineteenth century, the basic idea of 3D stereo came out. Since our two eyes are about 6.5 cm apart from our heads, each eye sees the scene we see as slightly different viewing angles, providing different views. After that, our brain can create a sense of depth in the scene based on two views from our eyes. Figure 1 illustrates the basic concept of 3D stereoscopic displays, where Z is the depth of the object to be recognized and D is the distance to the screen, in front of the screen (on the screen) (Tree), and the infinite distance (box), four objects are recognized. Most recent 3D displays are configured based on 3D stereo concepts, the main difference being in the way that two views, i.e., two views for the right eye and for the left eye, are separated.

Based on this principle, there are many ways to capture stereo photographs. Currently, the most popular methods are using the same two cameras and using a single stereo camera.

Figure 2 illustrates a basic method of using two cameras to capture a stereo photo graph. The two cameras are spaced apart by a certain distance. The two photographed images are simultaneously captured by two cameras. Thereafter, after development, the user sees the photo captured by the left camera as the user's left eye and the photo captured by the right camera as the user's right eye. And the user 's brain realizes the 3D space where the user captured the photograph.

A stereo camera is a type of camera with two or more lenses, each with a film frame or a separate image sensor for each lens. This allows the camera to simulate human binocular vision, thus providing the ability to capture three-dimensional images, a process known as stereophotography. Stereo cameras can also be used to create 3D pictures and stereo views for movies. The distance between the lenses in a stereo camera (axis-to-axis distance) is defined by how many three-dimensional numbers are desired and how far the target is located.

All of these methods require two lenses, and users must be professional photographers. And, due to hardware limitations on mobile devices, ordinary users with mobile devices with a single camera can hardly capture 3D photographs. If mobile device users want to capture stereo 3D photos, they need to capture two photos of the same object. First, a photo of an object is picked up. Thereafter, the camera is slightly moved to the left or right to capture the second photo. In the remainder of this disclosure, the first port is for the left eye and the second port is for the right eye. Finally, the two pictures are composited into a stereo 3D image by computing. Users spend a lot of time and energy in creating posts, but often the effects of 3D photos are not enough. Since they photograph two photos too subjective, the left image and the right image hardly match, and the 3D effect can not be realized. This disclosure proposes a method of capturing a stereo 3D image using a single camera for a mobile device. The system will assist the cameraman to accurately capture the left and right images and to obtain a better effect on the 3D pictures to simplify post generation.

Creating a three-dimensional illusion is entirely dependent on the fact that our eyes are separated by a certain distance. If each eye is looking at the same image shot from a slightly different angle, when our brain synthesizes the images, the synthesized image will show three dimensions. In accordance with this principle, the present invention provides a method for capturing stereo 3D images using a single camera. The camera will capture each of the left image and the right image. After the camera captures the left image, the system provides some prompts for the best position of the right image for the users. Users can accurately capture the right image for compositing to create a stereo image according to the prompts. Thus, the present invention helps to solve the problem of the method of providing some prompts for the position of the right image to be synthesized into a stereo 3D image.

A related art, US20100316282, discloses a method of generating a 3D image based on information on position / direction changes between a first picture and a second picture, and between a first picture and a second picture.

The present invention discloses a method of capturing a 3D image using a single camera. To capture a stereoscopic image, an image processing function is added to the mobile device with a single camera to match the left and right image feature points.

When the mobile device captures the left image, the system will extract the feature points of the left image. Then, when the mobile device moves to shoot the right picture, the system will extract the feature points of the right picture. Hereinafter, it should be noted that the right picture specifies the picture the camera is displaying on the display, while the right image specifies the image picked up by the camera to be synthesized in the stereo 3D image. Also, "capture" is used when capturing a picture, and "capturing" is used when a picture is displayed.

Thereafter, the system compares the minutiae point map of the right picture with the minutiae map of the left image to analyze the object size, using the minutia matching method based on bidirectional maximum correction and time lag constraints. If the object sizes in the two maps are the same, this suggests that the viewing distances for both are the same. If the object sizes in the two maps are different, the camera must move until the objects in the two maps are the same. The system also compares the vertical disparity between feature point maps. In addition, the camera must be moved and rotated to offset the vertical disparity. Finally, users will be able to capture the right image for accurate compositing into a stereo 3D image.

According to one aspect of the present invention, there is provided a method of generating a 3D image using a single camera, the method comprising: capturing a first image as a right or left image at a first position by a single camera; Extracting feature points of the first image; Capturing a picture to find a second image as the other image in a position different from the first position; Extracting feature points of a picture; Comparing the feature points of the first image and the picture; Generating two 3D cursors, one of the cursors representing a target position of the second image and the other representing a current position of the camera; Displaying two 3D cursors on a picture; Capturing a second image when the cursor indicating the current position completely overlaps the cursor indicating the target position by moving and rotating the camera; And synthesizing the first and second images to produce a 3D image.

According to another aspect of the present invention there is provided a system for generating a 3D image using a single camera, the system comprising: means for capturing a first image as a right or left image in a first position by a single camera; Means for extracting feature points of the first image; Means for photographing a picture to find a second image as the other image in a position different from the first position; Means for extracting feature points of a picture; Means for comparing feature points of a first image and a picture; Means for generating two 3D cursors, one of the cursors representing a target position of the second image and the other representing a current position of the camera; Means for displaying two 3D cursors on a picture; Means for capturing a second image when the cursor representing the current position completely overlaps the cursor representing the target position by moving and rotating the camera; And means for compositing the first and second images to produce a 3D image.

These and other aspects, features, and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
1 is an exemplary diagram illustrating the concept of 3D stereoscopic displays.
Figure 2 is an exemplary diagram illustrating a basic method of using two cameras to capture a stereo photo graph.
Figures 3A-3E illustrate an exemplary flow chart illustrating steps for capturing two images using a single camera in accordance with an embodiment of the present invention.
4A and 4B illustrate exemplary flow charts illustrating the main steps of a 3D image capture system for a mobile phone in accordance with an embodiment of the present invention.
5 is an exemplary flow chart illustrating an exemplary block diagram of a system according to an embodiment of the invention.

In the following description, various aspects of an embodiment of the present invention will be described. For purposes of explanation, various specific configurations and details are set forth in order to provide a thorough understanding. However, it will be apparent to one skilled in the art that the present invention may be practiced without the specific details disclosed herein.

The present invention focuses on the prompts of the second image capture to synthesize a stereo 3D image when users use a single camera. In general, it relates to a 3D image capture system that utilizes feature point matching methods to obtain position disparity data and parallax data between two images.

When the mobile device captures the left image, the system will extract the feature points of the left image so that the feature points of the right image for the stereo 3D image are inferred. In addition, the system will provide some prompts for the best position of the right image for the users. Users can accurately capture the right image for compositing into the stereo image according to the position data. When the user captures the right image, there are two 3D cursors on the screen. One represents the target position of the right image and the other represents the current position of the camera. When the two 3D cursors overlap, the user will capture the right image for accurate compositing into the stereo 3D image.

The steps of capturing two images using a mobile phone having a camera are described as follows according to FIGS. 3A to 3E.

1. A user captures a first image using a mobile phone having a camera (Fig. 3A).

2. Three depth information icons are displayed on the screen. The three icons represent "front of screen", "on screen" or "behind screen". The user selects one of three icons based on what effect the user desires for the parallax of the two images (Fig. 3B).

3. The user moves the mobile phone to the right to find a view of the right image (FIG. 3C).

4. Two 3D cursors are displayed on the screen. One representing the target position of the captured right image and the other representing the current position of the camera. The user makes an overlap of two 3D cursors through the previous and the rotation (FIG. 3D).

5. When the two 3D cursors overlap, the right image is captured (Figure 3e).

One of the system descriptions is for driving 3D cursors as follows.

1. The mobile device captures the left image.

2. The system picks up the left image as the first image and extracts the minutiae points of the first image.

3. Three depth information icons are displayed on the screen. The three icons represent "front of screen", "on screen" and "behind screen". The user selects one of three icons based on what effect the user desires for the parallax of the two images.

4. Move the mobile phone to the right to find the right image view.

5. The system extracts the feature points of the right picture to display on the display.

6. As is known, the latest 3D displays are based on 3D stereo concepts and the main difference is how to separate the two views for the left and right eyes, respectively. Thus, the system will analyze the parallax between the two pictures using a feature matching method.

7. Two 3D cursors are displayed on the screen to adjust the parallax between the two pictures.

8. The system compares the feature point map, that is, the set of feature points that represent the outer boundary of the object with respect to the left image, to the feature point map for the right picture, using the feature point matching method based on bidirectional maximum correction and time- The system analyzes the object sizes in both maps. If the object sizes in the two maps are the same, this suggests that the viewing distances for both pictures are the same

9. Two 3D cursors are displayed on the screen. If the size of each of the two 3D cursors is different, the user must move the camera forward or backward until both cursors are the same size. Thereby, the viewing distances for both the left image and the right image will be the same.

10. The system compares the vertical disparity between feature point maps.

11. Two 3D cursors are displayed on the screen to cancel the vertical disparity through previous and rotation.

12. When the two 3D cursors overlap, the right image is captured. The system captures the right image as the second image. The system then correctly composites the first and second images within the 3D stereo image.

Flow charts illustrating the main steps of a 3D image capture system in a mobile phone are shown in Figures 4A and 4B. Start at step 401. In step 403, the user captures the left image by a single camera. At step 405, the 3D image capture system extracts the feature points of the left image. At step 407, the 3D image capture system displays three depth information icons on the screen to suggest 3D effects to the user. The three icons represent "front of screen", "on screen" and "behind screen". In step 409, the user selects one of three icons to obtain the desired 3D effect. In step 411, the user moves the camera to find the view of the right image. In step 413, the 3D image capture system extracts the feature points of the right picture displayed on the display. In step 415, the 3D image capture system analyzes the parallax between the left image and the right picture by comparing their feature point maps. At step 417, the 3D image capture system displays two 3D cursors for the time difference adjustment in the display. One of them represents the target position of the right image and the other represents the current position of the camera. In step 419, if the depth effect is satisfactory, the process proceeds to step 421. In step 419, if the depth effect is not satisfactory, the process returns to step 411. In step 421, the 3D image system analyzes the object sizes in both the left image and the right picture using the feature point map. In step 423, the 3D imaging system displays the 3D cursors on the screen for the viewing distance of both the left and right pictures. In step 425, if the sizes of the two 3D cursors are equal, the process proceeds to step 427. In step 425, if the sizes of the two 3D cursors are not equal, the process returns to step 411. In step 427, the 3D imaging system compares the vertical disparity between the left image and the right picture using the feature point map. In step 429, the 3D imaging system displays two 3D cursors so that the user can offset the vertical disparity through previous movement and rotation of the camera. At step 431, if the two 3D cursors overlap, the process proceeds to step 433. At step 431, if the two 3D cursors do not overlap, the process returns to step 411. At step 433, the user captures the right image at the position where the two cursors overlap. At step 435, the 3D imaging system synthesizes the left and right images to produce a 3D image. Then, at step 427, the process proceeds to termination.

5 illustrates an exemplary block diagram of a system 510 in accordance with one embodiment of the present invention. The system 510 may be a mobile phone, a computer system, a tablet, a portable game, a smart phone, or the like. The system 510 includes a central processing unit (CPU) 511, a camera 512, a storage device 513, a display 514 and a user input module 515. The memory 516, such as RAM (Random Access Memory), may be connected to the CPU 511, as shown in FIG.

Camera 512 is an element for capturing left and right images by a single lens. The CPU 511 processes the steps of the 3D image capture system, as described above.

Display 514 is configured to visually present text, images, video, and any other content to a user of system 510. [ Display 514 may apply any type compatible with 3D content.

The storage device 513 is configured to store software programs and data for the CPU 511 to drive and operate the process to generate a 3D image as described above.

The user input module 515 may include keys or buttons for inputting characters or commands and may also include functions for recognizing characters or commands by keys or buttons. The user input module 515 may be omitted from the system depending on the application used in the system.

The present invention can be applied to mobile phones having cameras, such as mobile phones, tablets, and the like. Because the multi-view ports are essentially a series of adjacent stereo ports, similar multi-view ports as well as stereos can be imaged by similar hard air and software configurations.

These and other features and advantages of these principles may be readily ascertained by one of ordinary skill in the art based on the teachings herein. The teachings of the present principles may be implemented in various forms of hardware, software, firmware, special purpose processors, or any combination thereof.

More preferably, the teachings of the present principles are implemented as a combination of hardware and software. The software may also be implemented as an application program implemented in a type implemented on a program storage unit. The application program may be updated by a machine including any suitable architecture and executed by the machine. Preferably, the machine is implemented on a computer platform having hardware, such as one or more central processing units ("CPU"), random access memory ("RAM"), and input / output ("I / O" do. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be part of the microinstruction code, part of the application program, or any combination thereof, which may be executed by the CPU. In addition, several other peripheral units may be connected to a computer platform, such as an additional data storage unit.

It is to be understood that the actual connections between system components or process functional blocks may differ depending on the manner in which the principles are programmed, since some of the constituent system components and methods depicted in the accompanying drawings are preferably implemented in software I will also understand. Given the teachings herein, those skilled in the art will be able to contemplate these and similar implementations or configurations of these principles.

Although the illustrated embodiments are described herein with reference to the accompanying drawings, it is to be understood that these principles are not limited to these precise embodiments and that various changes and modifications may be effected to the person skilled in the art without departing from the scope or spirit of the principles As will be understood by those skilled in the art. All such changes and modifications are intended to be included within the scope of the present principles as set forth in the appended claims.

Claims (10)

A method of generating a 3D image using a single camera,
Capturing a first image as a right image or a left image by a single camera in a first position;
Extracting feature points of the first image;
Shooting a picture to find the second image as a different side image at a position different from the first position;
Extracting feature points of the picture;
Comparing the first image and the feature points of the picture;
Generating two 3D cursors, one of the cursors representing a target position of the second image and the other representing a current position of the camera;
Displaying two 3D cursors on the picture;
Capturing the second image when the cursor indicating the current position completely overlaps the cursor indicating the target position by moving and rotating the camera; And
And combining the first and second images to produce a 3D image. The method according to claim 1,
Wherein the comparing comprises analyzing the vertical disparity and the object size in the feature point map of the first image and the picture,
Wherein the feature point map displays feature points at an outer boundary of the object in each of the first image and the picture. 3. The method of claim 2,
Further comprising displaying three depth information icons to obtain a desired effect on the parallax of the first and second images,
The three depth information icons each indicate "front of screen "," on screen ", and &
Wherein the position of the target 3D cursor is determined to obtain a 3D effect of the selected depth icon. 4. The method according to any one of claims 1 to 3,
Wherein the viewing distances for the first position and the position are the same when the sizes of the two 3D cursors are the same and the sizes of the two 3D cursors are the same by moving the camera forward or backward, How to create an image. 5. The method according to any one of claims 1 to 4,
Wherein the vertical disparity between the first image and the picture is canceled by prior and rotating the camera. A system for generating a 3D image using a single camera,
Means for capturing a first image as a right image or a left image by a single camera in a first position;
Means for extracting feature points of the first image;
Means for photographing a picture to look for a second image as a different side image in a position different from the first position;
Means for extracting feature points of the picture;
Means for comparing minutiae points of the first image and the picture;
Means for generating two 3D cursors, one of the cursors representing a target position of the second image and the other representing a current position of the camera;
Means for displaying two 3D cursors on the picture;
Means for capturing the second image when the cursor representing the current position completely overlaps the cursor representing the target position by moving and rotating the camera; And
And means for compositing said first and second images to produce a 3D image. The method according to claim 6,
Wherein the comparing means comprises means for analyzing the vertical disparity and the object size in the feature point map of the first image and the picture,
Wherein the feature point map displays feature points at an outer boundary of the object in each of the first image and the picture. 8. The method of claim 7,
Further comprising means for displaying three depth information icons to obtain a desired effect on the parallax of the first and second images, wherein the three depth information icons are respectively referred to as "before the screen,"" Screen back "is displayed,
Wherein the position of the target 3D cursor is determined to obtain a 3D effect of the selected depth icon. 9. The method according to any one of claims 6 to 8,
Wherein the viewing distances for the first position and the position are the same when the sizes of the two 3D cursors are the same and the sizes of the two 3D cursors are the same by moving the camera forward or backward, A system for generating images. 10. The method according to any one of claims 6 to 9,
Wherein the vertical disparity between the first image and the picture is canceled by prior and rotating the camera.

Images