KR20160002510A - Coordinate Calculation Acquisition Device using Stereo Image and Method Thereof - Google Patents

Abstract

The present invention relates to a device and method for obtaining a photographing coordinate using a stereo image. The device of the present invention comprises: an image preprocessing unit for extracting each feature point from a plurality of photographing images in which the same object is photographed and preprocessing the same; a reference image generation unit for matching the photographing images by using the extracted feature point so as to generate one reference image; a coordinate matching unit for matching a pixel coordinate for the feature point of the reference image with an actual three-dimensional coordinate corresponding to the inputted feature point of the object; and an actual ground coordinate conversion unit for converting the pixel coordinate for the feature point of the reference image into an actual ground coordinate. According to the present invention, by introducing the device and method for obtaining the photographing coordinate using the stereo image, a coordinate of a photographing position is calculated based on an image so as to recognize a position of a user who photographs the image and to recognize the current position of a photographer even in the underground or an indoor space where a global positioning system (GPS), which is a conventionally representative method for recognizing the position, is difficult to be applied.

Description

Technical Field [0001] The present invention relates to a coordinate acquisition apparatus and a coordinate acquisition method using stereo images,

[0001] The present invention relates to an apparatus and method for acquiring a photographing coordinate using a stereoscopic image, and more particularly, to an apparatus and method for acquiring a photographing coordinate using a stereoscopic image, The present invention relates to a coordinate acquisition apparatus and a method thereof.

The location based service (LBS) is a method of locating location information using the mobile communication network and IT technology in a comprehensive manner and providing services based on the user's location information. The LBS includes a cellular system using a mobile communication base station and a GPS system utilizing a satellite navigation device. The cell type developed and distributed around the year 2000 identifies the base station connected to the subscriber based on the base station. Although the error range is not large due to the base stations set up at intervals of several tens of meters in the dense population area, an accurate range can not be obtained due to an error range of several km depending on the area. However, the cell method has an advantage that it can grasp the position of the indoor and the underground, which is a limitation of the GPS method.

On the other hand, the GPS method is a method in which a chip built in a user terminal receives location information transmitted from a satellite and notifies the base station of the received location information. In this method, the error range is several tens of meters, and it is possible to track the position more precisely than the cell type. However, due to the nature of the satellite signal, it is almost impossible to use it in high-rise buildings or indoors where reflection is good.

Therefore, beacons that can identify the position in the room also appeared. The beacon is a technology that uses wireless communication such as wireless sensor and Bluetooth low-power technology (BLE) to apply the localization technology. By analyzing the received intensity of each installed beacon, the position can be grasped and the position can be grasped precisely within an error range of about 5 cm within a range of 50 m. However, there is a limitation that a beacon device must be installed in order to determine the position using a beacon.

Using such a positional information grasping technique, a function of informing the user of the photographed image and the location information of the photographed image is widely utilized. The GPS function installed on the user terminal is mainly used. When the GPS setting mode is activated and the photograph is taken before the photographing, the position, the latitude and the longitude on the administrative area of the photographing place can be confirmed. Through this information, the position of the user who photographed the photograph can be confirmed. However, it is difficult to grasp the location information in the indoor or underground because the method of acquiring the location information using the GPS uses the satellite signal. Further, there is a disadvantage in that a user terminal can use a GPS signal only if it has a chip capable of receiving the GPS signal.

Therefore, there is a need for a method of locating the user in the room or underground. There is also a need for a method by which a user terminal can verify its location without having a separate device.

The technology that is the background of the present invention is disclosed in Korean Patent No. 10-0586815 (published on June, 2006).

An object of the present invention is to provide an apparatus and method for acquiring a coordinate of a photographing position using a stereo image, which is capable of calculating coordinates of a photographing position based on an image and determining the position of a user who has photographed the image.

According to an aspect of the present invention, there is provided an apparatus for obtaining a coordinate of a stereo image, the apparatus comprising: an image preprocessing unit for extracting and pre-processing feature points of a plurality of photographed images of the same object; A coordinate matching unit for matching the pixel coordinates of the minutiae of the reference image with the actual three-dimensional coordinates corresponding to the minutiae points of the input object, a reference image generating unit for generating a single reference image by matching the plurality of photographed images, And an actual terrain coordinate transformation unit for transforming pixel coordinates of feature points of the reference image into actual terrain coordinates.

The image preprocessing unit may include a feature point extracting unit that extracts feature points from the photographed image, an image correcting unit that corrects the photographed image based on a correction condition input from a user, And an inner facial expression element extracting unit for extracting an inner facial expression element of the facial expression.

The reference image generating unit may generate one reference image by matching the plurality of preprocessed images with each other through the SIFT algorithm, and extract pixel coordinates of the feature points of the reference image.

The coordinate matching unit can receive the actual three-dimensional coordinates of the point corresponding to the minutiae of the object through the total station.

The apparatus may further include a photographing position calculator for calculating a current position of the photographer who has taken the new image by applying the actual ground coordinates of the feature points of the stored reference image to the DLT algorithm, .

According to another aspect of the present invention, there is provided a method of acquiring a plurality of photographed images by extracting and pre-processing feature points of a plurality of photographed images of the same object, Matching the pixel coordinates of the minutiae points of the reference image with the actual three-dimensional coordinates corresponding to the input minutiae points of the input image, Into actual ground coordinates.

Therefore, according to the present invention, by using the photographing coordinate acquisition apparatus and method using the stereo image, coordinates of the photographing position can be calculated based on the image, and the position of the user who photographed the image can be grasped. It is difficult to apply GPS, but it is possible to grasp the current position of the photographer in the room.

1 is a block diagram illustrating a configuration of a photographing coordinate acquisition apparatus using a stereo image according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of an image preprocessing unit of a photographing coordinate acquisition apparatus using a stereo image according to an embodiment of the present invention.
3 is a flowchart illustrating a method of acquiring a photographing coordinate using a stereo image according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating a process of preprocessing an image in a method of acquiring a photographing coordinate using a stereo image according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 5 is a view for explaining the step S320 shown in FIG.
6 is a view for explaining steps S330 and S340 shown in FIG.
7 is a view for explaining a process of calculating a current position of a photographer according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, the configuration of the imaging coordinate acquisition apparatus 100 using a stereo image according to an embodiment of the present invention will be described in more detail with reference to FIG. 1 and FIG.

 1 is a block diagram illustrating a configuration of a photographing coordinate acquisition apparatus using a stereo image according to an embodiment of the present invention. 2 is a block diagram illustrating a configuration of an image preprocessing unit of a photographing coordinate acquisition apparatus using a stereo image according to an embodiment of the present invention.

1, the imaging coordinate acquisition apparatus 100 using a stereo image includes an image preprocessing unit 110, a reference image generating unit 120, a coordinate matching unit 130, and an actual terrain coordinate converting unit 140 And may further include a photographing position calculation unit 150.

The image preprocessing unit 110 corrects a plurality of photographed images of the same object and extracts characteristic points of each photographed image. 2, the image preprocessing unit 110 includes a feature point extracting unit 111, an image correcting unit 113, and an inner facial expression extracting unit 115.

The feature point extraction unit 111 extracts, as feature points, points that are easy to identify in the captured image.

The image correcting unit 113 corrects the photographed image under the correction conditions input from the user.

The internal facial expression extraction unit 115 extracts internal facial expression elements of the photographed image such as the principal point, focal length, distortion amount, lens position, rotation information, and scale.

The reference image generating unit 120 matches the plurality of preprocessed images captured by the image preprocessing unit 110 into a single reference image. At this time, a Scale Invariant Feature Transform (SIFT) algorithm is used to local-patch each feature point to extract a feature vector.

The coordinate matching unit 130 receives pixel coordinates for the minutiae of the reference image and actual three-dimensional coordinates corresponding to the minutiae through the total station.

The total station is an instrument that integrates electronic cadodlite (an instrument for observing the angle) and an electric-optical instrument (EDM, a distance measuring instrument) into a single instrument.

When the light source of the total station irradiates the light after collimating the prism, the light is immediately reflected upon arriving at the prism and returned to the light wave meter. This is used to measure the distance from the center of the total station body to the prism. After measuring the distance and position of one point, turn the total station body to the left and right to measure the distance and position of another point, and observe the angle between the two points.

The actual ground coordinate transforming unit 140 reconstructs the relationship between the pixel coordinate of the reference image matched by the coordinate matching unit 130 and the actual three-dimensional coordinate, and sets the pixel coordinates of the reference image as actual ground coordinates And stores it.

When a new image of the object included in the reference image is input, the photographing position calculation unit 150 calculates the position of the photographing user who has photographed the new image by applying the actual ground coordinates of the feature points of the stored reference image to the DLT algorithm .

Hereinafter, a method of acquiring a photographing coordinate using a stereo image according to an embodiment of the present invention will be described in more detail with reference to FIG. 3 through FIG.

3 is a flowchart illustrating a method of acquiring a photographing coordinate using a stereo image according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a process of preprocessing an image in a method of acquiring a photographing coordinate using a stereo image according to an embodiment of the present invention. Referring to FIG.

First, as shown in FIG. 3, the imaging coordinate acquisition apparatus 100 using a stereo image performs preprocessing to generate a reference image necessary for calculating a photographing position (S310).

The shooting coordinate acquisition apparatus 100 using the stereo image may perform an operation of improving the image quality if necessary to enhance the efficiency of image matching before extracting the feature points. The image quality is improved by applying image processing techniques to match images with various image qualities due to season, weather, etc. for image matching. At this time, it is possible to apply the histogram equalization (HE) method to improve the image quality by obtaining the transformation function using the histogram that characterizes the digital image using the image processing technique and expanding the dynamic range of the intensity.

As shown in FIG. 4, the feature point extracting unit 111 extracts, as feature points, a point, such as a corner point of an object photographed from the photographed image, that is easy to identify (S311). The extracted feature points are utilized in a process of finding a matching point that matches each other among a plurality of photographed images.

The image correcting unit 113 corrects the photographed image according to the correction condition input from the user among the extracted feature points from the feature point extracting unit 111 (S313). The correction condition is a feature point serving as a reference of correction, and may include a straight line or a curved outline representing the shape of the shooting object, a corner of the shooting object, or the like.

In addition, the image correction unit 113 removes the geometric distortion of the image so that the accurate position can be grasped.

Next, the inner facial expression extraction unit 115 extracts inner facial expression elements of the captured image using the extracted feature points from the feature extraction unit 111 (S315). The inner facial elements include the position of the principal point, the focal length of the camera, the lens distortion coefficient, the lens position, the rotation information, and the scale. The extracted inner facial element is used to obtain accurate pixel coordinates of the photographed image.

After the preprocessing step S310 of the image is finished, the reference image generator 120 generates a reference image by matching the plurality of photographed images using the extracted feature points (S320).

The reference image generating unit 120 generates a reference image as shown in FIG. 5 by matching the plurality of preprocessed images.

FIG. 5 is a view for explaining the step S320 shown in FIG.

5, when two captured images of the same object are respectively referred to as a left image and a right image, the reference image generating unit 120 uses the SIFT algorithm to match parts of the left image and the right image that match each other . We use Sparse Matching and Dense Matching as matching methods. The reference image generator 120 extracts feature points matching each other among the feature points of the left and right images using the SIFT algorithm and stores the matching pairs of the extracted feature points in the same row to generate one reference image . When the extracted feature points are (x1, y1) and the pixel coordinates are (x1 ', y1') in the left image, the coordinates stored in the reference image are (x1, y1, x1 ', y1 ')to be. The coordinates of all the feature points extracted in this way are stored as a matching pair in the reference image.

Next, the coordinate matching unit 130 matches the feature point of the generated reference image with the actual three-dimensional coordinates of the object corresponding to the feature point (S330). Extracts pixel coordinates for the minutiae of the generated reference image, and matches the pixel coordinates and the corresponding actual three-dimensional coordinates. The actual three-dimensional coordinates are the actual three-dimensional coordinates of the minutiae points of the actual object obtained using the total station. The coordinate matching unit 130 restores the relationship between the pixel coordinates of the reference image and the actual three-dimensional coordinates.

6 is a view for explaining steps S330 and S340 shown in FIG.

6, in step S330, the coordinate matching unit 130 matches the pixel coordinates of the minutiae of the reference image with the actual three-dimensional coordinates (x ', y', z ') of the input object, Restore the relationship between two coordinates.

In operation S340, the actual terrain coordinate transforming unit 140 transforms the pixel coordinates of the reference image into actual terrain coordinates using the pixel coordinates and the actual three-dimensional coordinates of the minutiae points of the reference image reconstructed in operation S330. Then, the actual terrain coordinate conversion unit 140 stores the actual terrain coordinates of the converted object in the cloud server. That is, the photographer can store the coordinate data stored in the server in the cloud server so as to use the coordinate data stored in the server. As illustrated in FIG. 6, the real ground coordinate transforming unit 140 matches the pixel coordinates of the minutiae of the reference image with the actual three-dimensional coordinates (x ', y', z ') of the input object. The real terrestrial coordinate transforming unit 140 matches the real three-dimensional coordinates of the object corresponding to the pixel coordinates of the minutiae of the reference image using the DLT technique. At this time, the real terrestrial coordinate transforming unit 140 performs the spatial resizing process and the spatial intersection process for generating the actual ground coordinates.

That is, the real ground coordinate transforming unit 140 maps the characteristic point of the reference image to a specific point of the real object through the spatial resizing process, and matches the actual three-dimensional coordinate of the specific point to the mapped point through the spatial intersection process.

According to the embodiment of the present invention, the pixel coordinates of the minutiae of the reference image are matched with the actual three-dimensional coordinates through steps S310 to S340, and the pixel coordinates of the minutiae of the reference image are converted into the actual ground coordinates, .

Hereinafter, a method of calculating the current position of the photographer himself / herself by using the actual ground coordinates of the object stored in the cloud server when the photographer photographs the object stored in the cloud server will be described.

When the photographing position calculation unit 150 receives a new photographing image including an object stored in the cloud server from the photographer, the photographing position calculation unit 150 applies the actual ground coordinates of the characteristic point of the reference image stored in the cloud server to the DLT algorithm, The current position of the photographer is calculated (S350).

The photographing position calculation unit 150 can calculate the position of the camera that photographed the image by using the actual ground coordinate data stored in the cloud server and the coordinates extracted from the image photographed by the photographer. At this time, the photographing position calculation unit 150 calculates the position of the camera using the DLT technique including the spatial resizing process and the spatial intersecing process.

In the spatial resizing process, the photographing position calculating unit 150 calculates the position and the DLT coefficient of the image by using the actual ground coordinates data and the coordinates extracted from the photographing image. In the spatial interpolation process, the photographing position calculation unit 150 generates a three-dimensional coordinate value by applying a DLT coefficient to a matching point between the image including the actual ground coordinate data and the photographed image, and grasps the position of the photographer.

7 is a view for explaining a process of calculating a current position of a photographer according to an embodiment of the present invention.

7, the photographing position calculation unit 150 calculates the photographing position of the reference image by comparing the actual ground coordinates (x1 ', y1') of the reference image stored in the cloud server with the image newly photographed by the photographer, Overlap based on common objects. (X1 ', y1', z1 ') of the minutiae stored in the cloud server and the pixel coordinates (x1', y1 ', z1' (Yco ', Zco') at which the newly photographed image is captured using the DLT technique using the image data "y1", "y1".

As described above, according to the embodiment of the present invention, the coordinates of the photographing position can be calculated based on the image by using the photographing coordinate acquiring device and the method using the stereo image, and the position of the photographing user can be grasped, It is difficult to apply GPS, which is a typical position detection method, but it can grasp the current position of the photographer in the room.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: Shooting coordinate acquisition device 110: Image preprocessing unit
111: feature point extracting unit 113:
115: inner facial expression element extracting unit 120:
130: coordinate matching unit 140: actual ground coordinate conversion unit
150: photographing position calculating section

Claims (10)

An image preprocessing unit for extracting and preprocessing feature points for a plurality of photographed images of the same object,
A reference image generation unit for generating a reference image by matching the plurality of photographed images using the extracted feature points,
A coordinate matching unit for matching the pixel coordinates of the minutiae points of the reference image with the actual three-dimensional coordinates corresponding to minutiae points of the input object, and
And an actual terrestrial coordinate converter for converting pixel coordinates of the minutiae of the reference image into actual terrain coordinates. The method according to claim 1,
The image pre-
A feature point extracting unit for extracting feature points from the captured image,
An image correction unit for correcting the photographed image based on a correction condition input from a user, and
And an inner facial expression element extracting unit for extracting an inner facial expression element of the captured image using the extracted feature points from the captured image. The method according to claim 1,
Wherein the reference image generating unit comprises:
A plurality of photographed images are matched to each other through a SIFT algorithm to generate one reference image, and pixel coordinates of the feature points of the reference image are extracted. The method according to claim 1,
Wherein the coordinate matching unit comprises:
Dimensional coordinate of a point corresponding to a feature point of the object through a total station. The method according to claim 1,
And a photographing position calculation unit for calculating a current position of the photographing person who has photographed the new image by applying the actual ground coordinates of the minutiae points of the stored reference image to the DLT algorithm when a new image including the object is input, For acquiring a photographing coordinate. Extracting and preprocessing feature points for a plurality of photographed images of the same object,
Generating a reference image by matching the plurality of photographed images using the extracted feature points,
Matching the pixel coordinates of the feature points of the reference image with the actual three-dimensional coordinates corresponding to the feature points of the input object, and
And converting pixel coordinates of feature points of the reference image into actual ground coordinates. The method according to claim 6,
Wherein the step of pre-
Extracting feature points from the photographed image,
Correcting the photographed image based on a correction condition input from a user, and
And extracting an internal facial expression element of the captured image using the extracted feature points from the captured image. The method according to claim 6,
Wherein the generating the reference image comprises:
A method of acquiring a captured image using a stereo image, the method comprising: generating a reference image by matching the plurality of preprocessed images with each other through a SIFT algorithm; and extracting pixel coordinates of the feature points of the reference image. The method according to claim 6,
The step of matching the coordinates comprises:
And acquiring actual three-dimensional coordinates of a point corresponding to a feature point of the object through a total station. The method according to claim 6,
Further comprising the step of calculating a current position of a photographer who has photographed the new image by applying the actual ground coordinates of the minutiae of the stored reference image to the DLT algorithm upon receiving a new image including the object, A method of acquiring shooting coordinates.

Images