KR101276314B1 - Matching point extraction system using feature point of lsh algorithm and thereof - Google Patents

Abstract

PURPOSE: A matching point extraction system and a method thereof using a feature point of a data inquiry result of an LSH (Locality-Sensitivity Hashing) algorithm are provided to obtain a homography value by extracting a feature point of an image stored in an image information DB and a converted feature point of a photographed image as a matching point. CONSTITUTION: A mobile terminal obtains image information through a camera and extracts a feature point of the obtained image (S100, S110). The mobile terminal determines the existence of the same information between the descriptor information stored in an image information DB and photographed image descriptor information using an LSH algorithm (S130). The mobile terminal converts a type of the feature point of the image (S140). A matching point extraction server extracts the feature point of the image stored in the image information DB and the converted feature point of the photographed image as a matching point (S150). [Reference numerals] (AA) Descriptor information stored in an image information DB; (S100) Take an image with a camera; (S110) Extract a feature point of the image; (S120) Extract an image descriptor; (S130) Identify a same image by using an LSH algorithm; (S140) Convert a type of feature point of the images; (S150) Extract a matching point between the images stored in the image information DB and the photographed images; (S160) Extract a homography value on the photographed images

Description

Matching point extraction system using feature point of LSH Algorithm and approximately.}

The present invention relates to a matching point extraction system and a method using a feature point of the data query result of the LS algorithm, and more particularly, to extract a matched image using the LSH algorithm from the image information registered in the image information DB In order to match the extracted image with the feature point of the image taken by the camera, a matching feature point is extracted from the captured image and registered in the format and image information DB of the feature point extracted from the image taken by the camera. Matching point extraction system and method using the feature points of data search result of LS algorithm that matches the format of feature points to extract the matching image by extracting the matching images and extracts the coordinates corresponding to each feature point It is about.

Augmented Reality combines virtual additional information with information from the real world to create a new environment.

Augmented reality displays real information and virtual information at the same time by registering desired virtual information on the real information.

There has been a movement to apply this technology in various fields of augmented reality, and in recent years, it is mainly focused on improving mobile device performance.

Existing augmented reality system recognizes the image to be tracked from the live image and obtains relative data from the image input device.

Based on this, we predict the shape and location of the virtual object and match it to the real image.

In order to easily recognize the image of the object and to calculate the relative position, a method using a marker in the form of a prescribed form is being used more and more.

However, there are many practical limitations in that the object to be augmented reality tracking must have a predetermined pattern of markers.

Therefore, there are various difficulties such as non-marker-based research that does not require such markers, but it is affected by external factors during tracking and exponential increase in tracking.

That is, a feature point for comparing image information registered in a DB configured at a remote location with image information captured by a camera configured in a user's smartphone, an image registered in a recognized server, and an image captured by a smartphone in a 3D space In order to extract a homograph value that changes to a two-dimensional plane form when viewed from one plane, the feature point format is different so that a duplicate operation that requires re-extracting the feature point on the image taken by the camera to obtain the homograph value is performed. There was no choice but to.

Therefore, the process for outputting the corresponding information is complicated, and the analysis and processing time is long when network traffic is generated, which makes it impossible to provide information quickly.

none.

Therefore, the present invention has been proposed in view of the above-described problems of the prior art, and an object of the present invention is to extract a matched image using the LSH algorithm from image information registered in the image information DB, and extract the extracted image and the camera. Extracts the feature points that are matched from the captured image for matching by using the feature points of the captured images, and matches the form of the feature points extracted from the images captured by the camera and the images registered in the image information DB. To match the format of the feature point to extract the image to extract the coordinates corresponding to each feature point to obtain a homograph value to eliminate the redundant work that needs to extract the feature point for the image taken by the camera.

In order to achieve the object of the present invention,

Matching point extraction system using the feature points of the data lookup results of the LS algorithm according to an embodiment of the present invention,

A camera unit 210 for acquiring image information,

An image feature point extracting unit 220 for extracting feature points of an image obtained through the camera unit;

A mobile terminal 200 including an image narrator extracting unit 230 for extracting an image descriptor photographed through the camera unit;

A video information DB 110 for storing video information;

The same image existence determination unit 120 to determine whether the same information exists by comparing the descriptor information stored in the image information DB and the photographed image descriptor information by using the LSH algorithm,

A feature point format converter 130 for converting a feature point format of an image photographed through the camera unit;

Matching point extraction unit 140 for extracting the feature point of the image stored in the image information DB and the converted feature point of the image photographed through the camera unit as a matching point;

And a matching point extracting server 100 including a homograph value extracting unit 150 for extracting a homograph value of an image photographed through a camera unit with reference to the matching point. Will be solved.

The matching point extraction system and method using the feature points of the data lookup results of the LS algorithm according to the present invention having the above configuration and operation,

Extracts a matching image from the image information registered in the image information DB using the LSH algorithm, and extracts the matching feature point from the captured image for matching using the feature point of the extracted image and the image captured by the camera. Extracts the matched format of the feature point extracted from the image captured by the camera and the image registered in the image information DB, and matches the format of the feature point to extract the matching image, and extracts the coordinates corresponding to each feature point. By obtaining the value, it provides the effect of eliminating the duplication of having to re-extract the feature points of the image captured by the camera.

1 is an overall configuration diagram of a matching point extraction system using the feature points of the data lookup results of the LS algorithm according to an embodiment of the present invention.
2 is a block diagram of a mobile terminal of a matching point extraction system using the feature points of the data query result of the LS algorithm according to an embodiment of the present invention.
3 is a block diagram of a matching point extraction server of the matching point extraction system using the feature points of the data query result of the LS algorithm according to an embodiment of the present invention.
4 is a flowchart illustrating a method of extracting a matching point using feature points of a data query result of an LS algorithm according to an embodiment of the present invention.

Matching point extraction system using the feature points of the data lookup results of the LS algorithm according to an embodiment of the present invention for achieving the above object,

A camera unit 210 for acquiring image information,

An image feature point extracting unit 220 for extracting feature points of an image obtained through the camera unit;

A mobile terminal 200 including an image narrator extracting unit 230 for extracting an image descriptor photographed through the camera unit;

A video information DB 110 for storing video information;

The same image existence determination unit 120 to determine whether the same information exists by comparing the descriptor information stored in the image information DB and the photographed image descriptor information by using the LSH algorithm,

A feature point format converter 130 for converting a feature point format of an image photographed through the camera unit;

Matching point extraction unit 140 for extracting the feature point of the image stored in the image information DB and the converted feature point of the image photographed through the camera unit as a matching point;

And a matching point extraction server 100 including a homography value extracting unit 150 for extracting a homograph value of the image photographed through the camera unit with reference to the matching point. .

At this time, the same image existence determination unit 120,

It is characterized by searching video using cvLSHQuery.

At this time, the feature point format conversion unit 130,

The feature point is searched using the CvSeq format, and the feature point is converted into a format for extracting a matching point that extracts the point coordinates from the CvSeq data using the CvSURFPoint format.

At this time, the matching point extraction unit 140,

Matching point of image point of image information and feature point of camera shot image by coordinates recognized by comparing the case where the stored values in sequence are 0 or more from extracted information to confirm matching image information using LSH It is characterized in that the extraction.

On the other hand, the matching point extraction method using the feature points of the data query results of the LS algorithm of the present invention,

An image capturing step (S100) of capturing an image with the camera unit 210 of the mobile terminal 200;

An image feature point extraction step (S110) of extracting feature points of an image acquired through the camera unit by the image feature point extractor 220;

An image narrator extracting step (S120) of extracting an image descriptor photographed through the camera unit by the image narrator extracting unit 230;

Comparing the descriptor information stored in the image information DB with the photographed image descriptor information by the same image existence determining unit 120 to determine whether the same information exists by using the LSH algorithm (S130);

An image feature point converting step (S140) for converting the feature point format of the image photographed by the camera unit by the feature point format converting unit 130;

Matching point extraction step (S150) for extracting the feature point of the image stored in the image information DB by the matching point extraction unit 140 and the transformed feature point of the image photographed through the camera unit as a matching point;

Homograph value extraction step (S160) for extracting a homograph value of the image taken through the camera unit by reference to the matching point by the homograph value extraction unit 150; characterized in that it comprises a.

Hereinafter, a description will be given in detail through an embodiment of a matching point extraction system and a method using a feature point of a data lookup result of the LS algorithm according to the present invention.

1 is an overall configuration diagram of a matching point extraction system using the feature points of the data lookup results of the LS algorithm according to an embodiment of the present invention.

2 is a block diagram of a mobile terminal of a matching point extraction system using the feature points of the data query result of the LS algorithm according to an embodiment of the present invention.

3 is a block diagram of a matching point extraction server of the matching point extraction system using the feature points of the data query result of the LS algorithm according to an embodiment of the present invention.

As shown in Figures 1 to 3, the system of the present invention,

A camera unit 210 for acquiring image information,

An image feature point extracting unit 220 for extracting feature points of an image obtained through the camera unit;

A mobile terminal 200 including an image narrator extracting unit 230 for extracting an image descriptor photographed through the camera unit;

A video information DB 110 for storing video information;

The same image existence determination unit 120 to determine whether the same information exists by comparing the descriptor information stored in the image information DB and the photographed image descriptor information by using the LSH algorithm,

A feature point format converter 130 for converting a feature point format of an image photographed through the camera unit;

Matching point extraction unit 140 for extracting the feature point of the image stored in the image information DB and the converted feature point of the image photographed through the camera unit as a matching point;

And a matching point extracting server 100 including a homograph value extracting unit 150 for extracting a homograph value of the image photographed through the camera unit with reference to the matching point.

The mobile terminal is a terminal that can communicate with the server to receive the matching point extraction information using the feature points of the data lookup results of the LS algorithm of the present invention, for example, a mobile phone, a smartphone, a tablet computer, a laptop, etc. May be a mobile communication terminal.

In addition, the matching point extraction server 100 may be connected to the mobile terminal by accessing the network through a wireless broadband network.

As shown in Figure 2, the mobile terminal 200,

A camera unit 210 for acquiring image information,

An image feature point extracting unit 220 for extracting feature points of an image obtained through the camera unit;

It is configured to include an image narrator extracting unit 230 for extracting an image descriptor photographed through the camera unit.

That is, image information is acquired through the camera unit 210, and the feature point of the image acquired through the camera unit is extracted by the image feature point extractor 220.

At this time, the image narrator extracting unit 230 is to extract the image descriptors taken through the camera unit at the same time.

On the other hand, as shown in Figure 3, the matching point extraction server 100,

A video information DB 110 for storing video information;

The same image existence determination unit 120 to determine whether the same information exists by comparing the descriptor information stored in the image information DB and the photographed image descriptor information by using the LSH algorithm,

A feature point format converter 130 for converting a feature point format of an image photographed through the camera unit;

Matching point extraction unit 140 for extracting the feature point of the image stored in the image information DB and the converted feature point of the image photographed through the camera unit as a matching point;

And a homography value extraction unit 150 for extracting a homograph value of the image photographed through the camera unit with reference to the matching point.

The same image existence judging unit 120 compares the descriptor information stored in the image information DB with the photographed image descriptor information using the LSH algorithm to determine whether the same information exists.

In this case, the feature point format conversion unit 130 of the present invention converts the feature point format of the image photographed through the camera unit.

That is, the format of the feature point extracted from the image captured by the camera unit and the format of the feature point for searching the image stored in the image information DB are matched and the coordinate values corresponding to each feature point are extracted. It is to achieve the purpose of obtaining.

Therefore, the feature point format is different, so that it is possible to provide an effect of reducing duplication of having to re-extract feature points for the image photographed by the camera unit in order to obtain a homograph value.

In detail, when searching an image using cvLSHQuery, the feature point is searched using the CvSeq format, and the feature point format is the Point2f format to extract the video match point.

It is responsible for the function of converting the format for extracting the matching point extracting the point coordinates using the CvSURFPoint format from the CvSeq data.

Thereafter, the matching point extracting unit 140 extracts the feature points of the image stored in the image information DB and the converted feature points of the image photographed through the camera unit as matching points.

Specifically, the sequence number in the extracted information when checking the matching image information using the LSH means the sequence number coordinates of the camera shooting feature, and the value corresponding to the sequence number of the extracted information when checking the matching image information using the LSH is assigned to the image information DB. Same as the sequence number of the stored image feature point coordinates.

The matching information is the same as the video ID stored in the video information DB and the extracted video ID when checking the matching video information using the LSH, and the value stored sequentially in the extracted information when checking the matching video information using the LSH is greater than or equal to '0'. In this case, the image feature points and the camera photographing feature points of the image information DB are extracted as matching points.

Thereafter, the homograph value extractor 150 extracts the homograph value of the image photographed through the camera unit with reference to the matching point.

Since the process of providing augmented reality using a homography value is a well known technique, detailed description thereof will be omitted.

4 is a flowchart illustrating a method of extracting a matching point using feature points of a data query result of an LS algorithm according to an embodiment of the present invention.

As shown in Figure 4, the matching point extraction method using the feature points of the data query results of the LS algorithm,

An image capturing step (S100) of capturing an image with the camera unit 210 of the mobile terminal 200;

An image feature point extraction step (S110) of extracting feature points of an image acquired through the camera unit by the image feature point extractor 220;

An image narrator extracting step (S120) of extracting an image descriptor photographed through the camera unit by the image narrator extracting unit 230;

Comparing the descriptor information stored in the image information DB with the photographed image descriptor information by the same image existence determining unit 120 to determine whether the same information exists by using the LSH algorithm (S130);

An image feature point converting step (S140) for converting the feature point format of the image photographed by the camera unit by the feature point format converting unit 130;

Matching point extraction step (S150) for extracting the feature point of the image stored in the image information DB by the matching point extraction unit 140 and the transformed feature point of the image photographed through the camera unit as a matching point;

Homograph value extraction step (S160) for extracting a homograph value of the image taken through the camera unit by reference to the matching point by the homograph value extraction unit 150; characterized in that it comprises a.

That is, when the image is photographed by the camera unit 210 of the mobile terminal 200 (S100), the feature point extractor 220 extracts the feature points of the image obtained through the camera unit and extracts the image narrator. By the unit 230, the image descriptor photographed through the camera unit is extracted (S120).

Thereafter, the same image existence determining unit 120 compares the descriptor information stored in the image information DB with the photographed image descriptor information by using the LSH algorithm, and determines whether the same information exists (S130). In operation 130, the feature point format of the image captured by the camera unit may be converted.

Thereafter, the matching point extracting unit 140 extracts the feature points of the image stored in the image information DB and the converted feature points of the image photographed through the camera unit as matching points (S150), and then, extracts the homograph value extracting unit 150. By extracting the homograph value of the image taken through the camera unit with reference to the matching point (S160), it is terminated.

Through the configuration and operation as described above, a matching image is extracted from the image information registered in the image information DB using the LSH algorithm, and photographed for matching using the feature points of the extracted image and the image taken by the camera. Extract the feature points that are matched in the image, match the format of the feature points extracted from the image captured by the camera with the format of the feature points to extract the matched image by searching the image registered in the image information DB, By extracting the corresponding coordinates to obtain a homograph value, it provides an effect of eliminating the duplication of the need to re-extract feature points for the image taken by the camera.

Those skilled in the art to which the present invention pertains as described above may understand that the present invention may be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the above-described embodiments are to be understood as illustrative in all respects and not restrictive.

The scope of the invention is indicated by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the invention. do.

100: matching point extraction server
200: mobile terminal

Claims (5)

In the matching point extraction system using the feature points of the data lookup results of the LS algorithm,
A camera unit 210 for acquiring image information,
An image feature point extracting unit 220 for extracting feature points of an image obtained through the camera unit;
A mobile terminal 200 including an image narrator extracting unit 230 for extracting an image descriptor photographed through the camera unit;
A video information DB 110 for storing video information;
The same image existence determination unit 120 to determine whether the same information exists by comparing the descriptor information stored in the image information DB and the photographed image descriptor information by using the LSH algorithm,
A feature point format converter 130 for converting a feature point format of an image photographed through the camera unit;
Matching point extraction unit 140 for extracting the feature point of the image stored in the image information DB and the converted feature point of the image photographed through the camera unit as a matching point;
And a matching point extraction server (100) configured to include a homography value extracting unit (150) for extracting a homograph value of an image photographed through a camera unit with reference to the matching point. Matching point extraction system using feature points of data search result of LS algorithm.
The method of claim 1,
The same image existence determination unit 120,
Matching point extraction system using the feature points of the data query result of the LS algorithm characterized by querying the image using cvLSHQuery.
The method of claim 1,
The feature point format conversion unit 130,
Matching points using the feature points of the data lookup result of the LS algorithm, characterized by querying the feature points using the CvSeq format, and converting the CvSeq data into a format for extracting the matching points using the CvSURFPoint format. Extraction system.
The method of claim 1,
The matching point extraction unit 140,
Matching point of image point of image information and feature point of camera shot image by coordinates recognized by comparing the case where the stored values in sequence are 0 or more from extracted information to confirm matching image information using LSH Matching point extraction system using the feature points of the data query results of the LS algorithm, characterized in that extracted by.
In the matching point extraction method using the feature points of the data query results of the LS algorithm,
An image capturing step (S100) of capturing an image with the camera unit 210 of the mobile terminal 200;
An image feature point extraction step (S110) of extracting feature points of an image acquired through the camera unit by the image feature point extractor 220;
An image narrator extracting step (S120) of extracting an image descriptor photographed through the camera unit by the image narrator extracting unit 230;
Comparing the descriptor information stored in the image information DB with the photographed image descriptor information by the same image existence determining unit 120 to determine whether the same information exists by using the LSH algorithm (S130);
An image feature point converting step (S140) for converting the feature point format of the image photographed by the camera unit by the feature point format converting unit 130;
Matching point extraction step (S150) for extracting the feature point of the image stored in the image information DB by the matching point extraction unit 140 and the transformed feature point of the image photographed through the camera unit as a matching point;
The homography value extraction step (S160) for extracting a homograph value of the image taken through the camera unit by reference to the matching point by the homograph value extraction unit 150; Matching point extraction method using feature points of data search results.

Images