KR100724889B1 - Apparatus and method for providing distance and 3-dimension image using stereo image in mobile terminal - Google Patents

Abstract

An apparatus and a method for providing a distance and a three-dimension image by using a stereo image in a mobile communication terminal are provided to obtain the stereo image by using one camera sensor without an additional camera sensor. The first image signal obtaining module(102) obtains the first image signal from a camera module. The second image signal obtaining module(104) obtains the second image signal, which has predetermined displacement variation with the first image signal, from the camera module. The first feature point extracting module(106) extracts at least one first feature point for a point selected by a user in the first image. A feature point matching module(108) performs matching for feature points to search for a point corresponding to the first feature point in the second image signal. The second feature point extracting module(110) extracts the second feature point from the second image. A distance measuring module measures a distance for the point selected by the user through displacement differences corresponding to position differences between the first and second feature points and points in which the first and second images are obtained.

Description

Apparatus and method for providing distance and 3D image using stereo image in mobile communication terminal {APPARATUS AND METHOD FOR PROVIDING DISTANCE AND 3-DIMENSION IMAGE USING STEREO IMAGE IN MOBILE TERMINAL}

1 is a block diagram of an apparatus for providing a distance and a 3D image using a stereo image according to an exemplary embodiment of the present invention

2 is a flowchart illustrating a process of providing a distance and a 3D image using a stereo image according to an exemplary embodiment of the present invention.

3 is a diagram illustrating a distance measuring principle using a stereo image according to an exemplary embodiment of the present invention.

4 is an exemplary view illustrating a distance measuring process using a stereo image according to an exemplary embodiment of the present invention.

5 is an exemplary view illustrating a 3D image providing process using a stereo image according to an exemplary embodiment of the present invention.

The present invention relates to a stereo image technology, and more particularly, to an apparatus and method for using stereo image in a mobile communication terminal.

 Recently, a stereo vision (STEREO VISION) technology capable of obtaining three-dimensional information using a two-dimensional image has been developed and widely used. Steriovision technology is one of the fields of computer vision for automating the distance capabilities of human vision systems. These stereo vision technologies are more effective and less constrained by the actual application environment than using ultrasound and lasers as light sources to measure distances as a function of light flight time and speed, resulting in areas such as medical imaging, factory automation, and mapping. It has been widely used in.

However, in general, a mobile communication terminal is equipped with one camera, and provides a 2D image using the same, and does not provide a service using a stereo vision technology. This is because the stereo vision technology needs to acquire stereo images using two camera modules to obtain three-dimensional information, but there are many limitations in applying such technology to the camera module system of a mobile communication terminal (mobile phone, PDA, etc.). .

In other words, due to the recent miniaturization of mobile communication cameras, it is not appropriate to mount two camera sensors to acquire stereo images, and even if two cameras are mounted, it is difficult to operate two cameras at the same time. There is this rising problem.

Accordingly, an aspect of the present invention is to provide an apparatus and method for providing distance and three-dimensional images using stereo images in a mobile communication terminal that can apply stereo vision technology to a camera of a mobile communication terminal.

In addition, the present invention is to provide a distance and three-dimensional image providing apparatus and method in a mobile communication terminal providing a distance and three-dimensional image using a stereo vision technology at a low cost by using a single camera.

According to an aspect of the present invention, there is provided a distance and three-dimensional image providing apparatus using stereo images in a mobile communication terminal, comprising: a first image signal obtaining unit obtaining a first image signal from a camera module, and A second image signal acquisition unit for acquiring a second image signal having a constant displacement difference from the first image signal, and a first image extracting at least one first feature point at a point selected by a user with respect to the first image; A feature point extracting unit, a feature point matching unit performing a feature point matching to find a point corresponding to a first feature point in the second image signal, and a second feature point extraction extracting second feature points from the second image as a result of the feature point matching And a displacement difference corresponding to a position difference between the first and second feature points and the point at which the first and second images are acquired. It includes a distance measuring unit for measuring the distance to the point selected by the user.

In addition, the present invention provides a distance and three-dimensional image providing method using a stereo image in a mobile communication terminal, the process of obtaining a first image signal and a second image signal having a predetermined displacement difference from the camera module, Extracting at least one first feature point with respect to a point selected by the user, performing feature point matching to find a point corresponding to the first feature point in the second image signal, and as a result of the feature point matching Extracting second feature points from an image, and using a displacement difference corresponding to a position difference between the first and second feature points and a point at which the first and second images were acquired, for the point selected by the user. Measuring the distance.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the same components in the drawings are represented by the same reference numerals and symbols as much as possible even if shown in different drawings. In addition, in describing the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 is a block diagram of a distance and 3D image providing apparatus using stereo images in a mobile communication terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the apparatus for providing a distance and a 3D image using stereo images in a mobile communication terminal according to an exemplary embodiment of the present invention may include a camera module 101, a first image acquirer 102, and a second image acquirer ( 104, a first feature point extractor 106, a feature point matcher 108, a second feature point extractor 110, a distance measurer 112, a 3D image generator 114, and a display 116. do.

The camera module 101 includes at least one camera sensor, photographs a subject with a predetermined displacement difference according to a user's request, and outputs a photographed image signal. For example, when the camera module 101 includes one camera sensor, the camera module 101 includes a sliding support for allowing the camera sensor to slide, and a constant displacement difference is obtained by photographing a subject while sliding the camera sensor by a predetermined interval through the sliding support. The first and second video signals having the output are output. In addition, when the camera module 101 includes two camera sensors, the two cameras are installed to have a predetermined interval, and the first and second image signals having a constant displacement difference are output by photographing a subject at the same time. do. At this time, two or more video signals are possible. However, the present invention will be described with an example of two video signals.

The first image acquisition unit 102 acquires any one of the first and second image signals output by the camera module 101, that is, the first image signal and converts the obtained first image signal into a first image signal. Output to the feature point extraction unit 106.

The second image acquisition unit 104 may output an image signal different from the image acquired by the first image acquisition unit 102, that is, the second image signal among the first and second image signals output by the camera module 101. The acquired second image signal is output to the feature point matching unit 108.

The first feature point extractor 106 extracts at least one or more first feature points from the first image output from the first image acquirer 102 and outputs the first feature points to the feature point matcher 108. For example, when the distance measurement is required, the first feature point extractor 106 extracts a reference first feature point that is a reference for distance measurement from the first image, and extracts 3D information from the first image when the 3D image is required. Extract a plurality of first feature points to obtain. In this case, when distance measurement is required, the first feature point extractor 106 selects a pixel that is a feature of the pixel from among the pixels of the first image to measure the distance of the desired point, Determine the coordinates. In addition, when the 3D image is required, the second feature point extractor 106 selects a plurality of pixels that may be characteristic of the 3D image among the pixels of the first image to obtain the 3D image of the area selected by the user. The coordinates of the selected pixels are determined.

The feature point matching unit 108 receives the second image signal obtained from the second image acquisition unit 104 and receives the first feature point extracted from the first feature point extractor 106. The feature point matching unit 108 performs feature point matching to find a point corresponding to the first feature point in the second image signal.

The second feature point extractor 110 extracts the second feature points from the second image acquired by the second image acquirer 104 as a result of the feature point match by the feature point matcher 108. In this case, the second feature point extractor 106 selects pixels of a point corresponding to the first feature point in the second image and determines coordinates of the selected pixel. The second feature point extractor 106 outputs coordinate information about the first feature point and the second feature point. In this case, when the distance measurement is required, the second feature point extractor 106 is provided with respect to the first feature point and the second feature point. The coordinate information is output to the distance measuring unit 112, and when the 3D image is required, the coordinate information about the plurality of first feature points and the plurality of second feature points is output to the 3D image generator 114.

The distance measuring unit 112 receives coordinate information about the first feature point and the second feature point when the distance measurement is required. When the coordinate measuring unit 112 receives coordinate information about the first feature point and the second feature point, the distance measurer 112 attempts to obtain a distance by using a displacement difference corresponding to a position difference between a point at which the first image and the second image are acquired. Measure and output the distance to the point.

When the 3D image is required, the 3D image generator 114 receives coordinate information about the plurality of first feature points and the plurality of second feature points. When the 3D image generator 114 receives coordinate information about the first feature points and the second feature points, the 3D image generator 114 uniformly uses the displacement difference corresponding to the position difference between the points where the first and second images are acquired. A three-dimensional image of the area is generated and output.

The display unit 116 may be configured as a liquid crystal display (LCD) or the like. The display unit 116 may display a distance with respect to a specific point measured by the distance measuring unit 112 or may be generated by the 3D image generating unit 114. A three-dimensional image of the area is displayed.

Hereinafter, the operation of the apparatus for providing distance and 3D image using stereo images in the mobile communication terminal configured as described above will be described in detail. 2 is a flowchart illustrating a process of providing a distance and a 3D image using a stereo image according to an exemplary embodiment of the present invention. Referring to FIG. 2, a distance and 3D image providing apparatus (hereinafter, referred to as a “distance and 3D image providing apparatus”) using a stereo image in a mobile communication terminal may include a stereo image (eg, a first displacement) having a constant displacement difference in step 202. And a second image). In this case, the apparatus for providing a distance and the 3D image may acquire the first and second image signals having a constant displacement difference by photographing a subject by sliding one camera by a predetermined interval. In addition, the first and second image signals having a certain displacement difference may be obtained by photographing a subject with two cameras installed at a predetermined interval.

As described above, when the first and second image signals are acquired, the distance and the 3D image providing apparatus determine whether to measure the distance or generate the 3D image according to a user selection in step 204. In this case, the determination of whether to measure the distance or generate the 3D image may be performed before the first and first image signals are acquired.

First, when the distance measurement is selected by the user, the distance and 3D image providing apparatus proceeds to step 206 and extracts a first feature point that is a reference for distance measurement from the first image. In this case, when a pixel that is a feature of the point is selected among the pixels of the first image by the user, the first feature point extractor 106 extracts the first feature point by determining coordinates of the selected pixel.

In operation 208, the distance and 3D image providing apparatus extracts the second feature point by matching the feature point of the first image and the second image. In this case, the feature point matching unit 108 performs feature point matching to find a point corresponding to the first feature point in the second image signal, and the second feature point extractor 110 performs the feature point matching by the feature point matching unit 108. The second feature point is extracted from the second image acquired by the second image acquirer 104.

When the second feature point is extracted, the distance and the 3D image providing apparatus measure the distance using the first feature point and the second feature point in step 210. In this case, the distance measuring unit 112 may determine a distance (first feature point) using a displacement difference corresponding to the position difference between the first feature point, the second feature point, and the point where the first image and the second image were acquired. Output the measured distance. In operation 212, the distance and the 3D image providing apparatus provide the user with the measured distance. In this case, the measured distance may be displayed on the display unit 116.

Referring to the distance measurement process in step 206 to step 212 in more detail through the screen, Figure 3 is an exemplary view illustrating a distance measurement process using a stereo image according to an embodiment of the present invention.

Referring to FIG. 3, FIG. 3A illustrates acquired first and second video screens. When the first and second images are acquired, the apparatus for providing a distance and the 3D image displays the first image 30 and the second image 32 on one screen as shown in FIG.

The user may select a point on which the user wants to measure the distance in the first image 30 and the second image 32 through the screen as shown in FIG. 3A. In this case, the distance and 3D image providing device may enlarge and display an image so that the user may select a desired point in units of pixels for more accurate distance measurement. After zooming in the image, a position such as an intersection point or a boundary line may be displayed. Determine and select the feature point.

For example, when the user selects a point (x) (not shown) for measuring distance in the first image 30, the apparatus for providing distance and three-dimensional images may display the user in the first image as shown in FIG. 3B. The first feature point P1 is extracted by selecting a feature point, such as an intersection point or a boundary line, as a feature point with respect to the selected point.

When the first feature point P1 is extracted, the distance and the 3D image providing apparatus perform feature point matching to find a point corresponding to the first feature point in the second image as illustrated in FIG. 3C, and as a result of the feature point match, The second feature point P2 is extracted from the two images. In this case, the distance and the 3D image providing apparatus may extract the second feature point P2 after enlarging the image for more accurate matching.

As described above, when the first feature point P1 and the second feature point P2 are obtained, the distance and the 3D image providing apparatus measure the distance using the first feature point P1 and the second feature point P2. . 4 is a diagram illustrating a distance measuring principle using a stereo image according to an exemplary embodiment of the present invention.

Referring to FIG. 4, reference numeral 30 denotes a first image, and reference numeral 32 denotes a second image. P1 is a first feature point of the first image, and P2 is a second feature point of the second image. Using stereo vision technology, the position of the specific point x selected by the user is calculated as in Equation 1 below.

In Equation 1, L is a displacement difference of the camera module, and f is a focal length. And dl and dr are the displacement difference of the image generated by the displacement of the camera module. According to Equation 1, the distance D to a specific point x selected by the user may be obtained by dividing the product of the displacement difference and the focal length of the camera module by the displacement difference of the total image.

As described above, when the distance is obtained, the distance and the 3D image providing apparatus display the distance obtained on the display unit 116 to provide the user with a distance to a specific point x.

Meanwhile, referring to the case where the distance measurement is selected by the user in step 204 of FIG. 2, in step 216, the apparatus for providing distance and the 3D image extracts a plurality of first feature points for generating the 3D image from the first image. do. In this case, the first feature point extractor 106 selects a region to be 3D restored by the user, and when the number of feature points to be used in the selected region is selected, the first feature point extractor 106 extracts as many first feature points as the selected number within the selected region. .

When the first feature points are extracted, the distance and 3D image providing apparatus extracts the second feature points by matching the plurality of first feature points with the second image (218). At this time, the feature point matching unit 108 performs feature point matching to find the points corresponding to the plurality of first feature points in the second image signal, and as a result of the feature point matching by the feature point matching unit 108, 2 Extract the feature points.

Then, the apparatus for providing distance and 3D image acquires 3D information in step 220. In operation 222, the distance and the 3D image providing apparatus provide a 3D image using the obtained 3D information. In this case, the 3D image may be displayed on the display unit 116 to be provided to the user.

Referring to the three-dimensional image providing process in step 216 to step 222 through the screen in more detail, Figure 5 is an exemplary view illustrating a three-dimensional image providing process using a stereo image according to an embodiment of the present invention. .

Referring to FIG. 5, FIG. 5A illustrates first and second image screens acquired for providing a 3D image. When the first and second images are acquired, the apparatus for providing a distance and the 3D image displays the first image 50 and the second image 52 on one screen as shown in FIG.

The user may select a region in which one of the first image 50 and the second image 52 is to be provided with the 3D image through the screen as shown in FIG. 5A. In addition, the user may select the number of feature points to be used within the selected area.

For example, a user selects an area for receiving a 3D image from the first image 50 and the second image 52, and selects feature points to be used in one of the selected regions (first image). When the number is selected, the distance and 3D image providing apparatus extracts the first feature points PS1 by the number of selected feature points in the selected area K1 as shown in FIG.

When the first feature points PS1 are extracted, the distance and the 3D image providing apparatus may perform feature point matching to find a point corresponding to the first feature points PS1 in the second image 52 as shown in FIG. As a result of feature point matching, second feature points PS2 are extracted from the second image. In this case, an area-based method may be used as the feature point matching method. When the region-based method is used, a position having a maximum variance is determined in the selected region of the first image 50, and the selected region of the second image 50 is common to the first image 50. Determine the position with the maximum displacement at. In this case, the apparatus for providing a distance and the 3D image may determine a position by setting a predetermined area as a window. As described above, when the position having the maximum displacement is determined in the selected regions of the first image 50 and the second image 50, the position having the maximum correlation and the position having the maximum correlation between the first feature point. Next, the second feature points PS2 of the second image 52 are extracted.

As described above, when the first feature points PS1 and the second feature points PS2 are obtained, the apparatus for providing a distance and the 3D image is three-dimensional by using the first feature point PS1 and the second feature point PS2. The information is obtained and the 3D image is reconstructed using a rendering technique. The restored 3D image is displayed on the display unit 116 and provided to the user.

In the above description of the present invention, specific embodiments have been described, but various modifications may be made without departing from the scope of the present invention. Therefore, the scope of the present invention should not be defined by the described embodiments, but should be determined by the equivalent of claims and claims.

As described above, the present invention has an effect of providing distance and three-dimensional image using stereo vision technology by applying stereo vision technology to a mobile communication terminal at a low cost.

In addition, the present invention requires a stereo image obtained from the two camera sensors in order to restore the distance, in the present invention has the effect of obtaining a stereo image with one camera sensor without adding a separate camera sensor to the existing mobile communication terminal. have.

Claims (14)

In the distance and three-dimensional image providing apparatus using a stereo image in a mobile communication terminal, A first video signal acquisition unit obtaining a first video signal from a camera module; A second image signal acquisition unit for acquiring a second image signal having a predetermined displacement difference from the first image signal from the camera module; A first feature point extractor for extracting at least one first feature point from a point selected by the user with respect to the first image; A feature point matching unit performing feature point matching to find a point corresponding to a first feature point in the second image signal; A second feature point extracting unit extracting second feature points from the second image as a result of the feature point matching; And a distance measuring unit configured to measure a distance to a point selected by the user by using a displacement difference corresponding to a position difference between the first and second feature points and the point at which the first image and the second image were acquired. An apparatus for providing a distance and a 3D image using a stereo image in a mobile communication terminal. The method of claim 1, And a display unit for displaying the measured distance from the first image and the second image. 3. The method of claim 1, The apparatus for providing distance and 3D image using stereo image in the mobile communication terminal, wherein the point selected by the user is a certain area. The method of claim 3, 3D image generation for generating a 3D image of a predetermined area selected by the user by using a displacement difference corresponding to a position difference between the first and second feature points and the point where the first image and the second image were acquired. An apparatus for providing a distance and a 3D image using a stereo image in a mobile communication terminal further comprising a unit. The method of claim 4, wherein And a display unit for displaying the first image and the second image and the generated image. 3. The method according to any one of claims 1 to 5, The camera module includes two camera sensors provided to have a predetermined interval, and outputs first and second image signals having a constant displacement difference by simultaneously photographing a subject from both cameras. Distance and 3D image providing device using the image. The method according to any one of claims 1 to 5, The camera module includes a camera sensor that is slidable, and moves the camera sensor through a sliding support by sliding the camera sensor by a predetermined interval, thereby photographing a subject and outputting first and second image signals having a constant displacement difference. Device for providing distance and 3D image using stereo image in communication terminal. In the distance and three-dimensional image providing method using a stereo image in a mobile communication terminal, Obtaining a first image signal and a second image signal having a predetermined displacement difference from the camera module; Extracting at least one first feature point with respect to the point selected by the user with respect to the first image; Performing feature point matching to find a point corresponding to a first feature point in the second image signal; Extracting second feature points from the second image as a result of the feature point matching; Measuring a distance to a point selected by the user by using a displacement difference corresponding to a position difference between the first and second feature points and a point at which the first image and the second image were acquired. Distance and three-dimensional image providing method using a stereo image in a mobile communication terminal. The method of claim 8, And displaying the distance measured with respect to the point selected by the user. 3. The method of claim 8, The point selected by the user is a certain area, the distance and three-dimensional image providing method using a stereo image in a mobile communication terminal. The method of claim 8, Generating a 3D image of a predetermined area selected by the user by using a displacement difference corresponding to a position difference between the first and second feature points and the point where the first image and the second image were acquired. Method for providing distance and three-dimensional image using a stereo image in a mobile communication terminal characterized in that. The method of claim 11, And displaying the image generated for the predetermined region selected by the user. 3. The method according to any one of claims 8 to 12, The camera module includes two camera sensors provided to have a predetermined interval, and outputs first and second image signals having a constant displacement difference by simultaneously photographing a subject from both cameras. Method for providing distance and 3D image using image. The method according to any one of claims 8 to 12, The camera module includes a camera sensor that is slidable, and moves the camera sensor through a sliding support by sliding the camera sensor by a predetermined interval, thereby photographing a subject and outputting first and second image signals having a constant displacement difference. Method for providing distance and 3D image using stereo image in communication terminal.

Images