KR20140054940A - Method for providing information - Google Patents

Abstract

The present invention relates to a method of providing information corresponding to a received query image. A method of providing information corresponding to an image by an information provision apparatus according to an embodiment of the present invention may include the steps of collecting a specific image and information on the specific image by the information provision apparatus; combining at least two points of a reference object included in the collected specific image and creating one representative feature by the information provision apparatus; indexing the created representative feature by the information provision apparatus; receiving a query image from a user terminal by the information provision apparatus; comparing a predetermined point of the query object included in the received query image with the representative feature by the information provision apparatus; and transmitting information mapped with the query image to the user terminal based on the comparison result by the information provision apparatus.

Description

{METHOD FOR PROVIDING INFORMATION}

The present invention relates to an information providing method, and more particularly, to a method of providing information corresponding to a received query image.

The camera is basic in the smart phone, and various sensors such as GPS, illuminance sensor, gyro sensor, etc. are recognized to recognize the current position and state of the user. Using these devices as a user interactive interface and various sensors can dramatically improve user convenience and realism.

Recently, research is being conducted on a technique for providing information on a specific area using a user terminal such as a smart phone.

Registration No. 10-0469825 discloses a method for calculating the number of times of inputting a search word inputted for a search request within a predetermined period independently of the searcher's location information to thereby calculate the popularity information of the region And a local popular information providing system.

In addition, in the above-mentioned patent, in extracting a search term as a local popular search term, a predetermined correction process is performed on the extracted search term, thereby avoiding duplicate exposure with all the popular search terms and increasing the extraction probability of the search term directly related to the region And a local popular information providing system are disclosed.

However, the method disclosed in Japanese Patent Application No. 10-0469825 is limited to a method of using a query as a query, and thus there is a limit to the inputted query. In addition, the method disclosed in Patent No. 10-0469825 does not disclose a technique for accurately and promptly providing information corresponding to a query.

Therefore, it is necessary to study the technology that diversifies the input query and provides the information corresponding to the query accurately and quickly.

An object of the present invention is to provide an information providing method for quickly recognizing a query image effectively and providing information corresponding to the query image.

According to an aspect of the present invention, there is provided a method of providing information corresponding to an image, the method comprising: collecting a specific image and information about the specific image; The information providing apparatus merging at least two points of the reference object included in the collected specific image to generate one representative feature; The information providing device indexing the generated representative feature; Receiving the query image from the user terminal; Comparing the representative feature with a predetermined point of the query object included in the received query image; And transmitting the information mapped to the query image to the user terminal based on the comparison result.

The information providing method according to an embodiment of the present invention can quickly and accurately provide information corresponding to a query image to a user.

1 is a block diagram illustrating an information providing method according to an embodiment of the present invention.
2 is a block diagram of an information providing apparatus according to an embodiment of the present invention.
3 to 4 are flowcharts showing an information providing method according to an embodiment of the present invention.
5 is a diagram illustrating an example in which information about a query image is provided through an information providing method according to an embodiment of the present invention.

Hereinafter, an information providing apparatus for performing an information providing method and an information providing method according to an embodiment of the present invention will be described with reference to the drawings.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising ", or" comprising "and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

1 is a block diagram illustrating an information providing method according to an embodiment of the present invention.

As shown in the figure, the information providing apparatus 100 may be connected to the user terminal 200 and the web server 300 through a network. The term " network " refers to a network that performs communication through a medium, and may include an Internet network, a wired / wireless communication network, a broadcasting network, and a telephone network.

The information providing apparatus 100 may be composed of one single server or a plurality of servers (for example, a server may be installed in each region).

The user terminal 200 can be a portable terminal, a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation device, an MP3 player, .

In addition, the web server 300 may be a single server or a plurality of servers (e.g., a server for each region). The web server 300 receives a query for local information (e.g., information including longitude and latitude) from the information providing apparatus 100 and provides the image corresponding to the local information query to the information providing apparatus 100 . Also, the web server 300 may receive a specific keyword query from the information providing apparatus 100 and provide an image to the information providing apparatus 100 in the specific keyword query.

The user can access the information providing apparatus 100 and register his / her information. In this case, the user may use a specific application to easily connect to the information providing apparatus 100. [

2 is a block diagram of an information providing apparatus according to an embodiment of the present invention.

The information providing apparatus 100 includes a receiving unit 110, a representative feature generating unit 120, an indexing unit 130, a database 140, a comparing unit 150, a transmitting unit 160, Lt; RTI ID = 0.0 > 170 < / RTI >

The receiving unit 110 can receive various information, data, and the like. For example, the receiving unit 110 may receive an image including an object. The image including the object may be an image received from the web server 300 or a query image received from the user terminal 200. The image received from the web server 300 may be an image corresponding to a query transmitted by the information providing apparatus 100 to the web server 300 (e.g., a query in the form of local information or a keyword). The image received from the web server 300 may be an image that should be collected in advance in order to construct the database 140 of the information providing apparatus 100. In this specification, the query image refers to an image transmitted by the user terminal 200 as a query to the information providing apparatus 100 for a specific information request.

In this specification, for convenience of explanation, an image collected in advance for establishing the database 140 of the information providing apparatus 100 is referred to as a 'reference image,' and an image received from the user terminal 200 for requesting information is referred to as' Query image '. The object included in the reference image is referred to as a 'reference object', and the object included in the query image is used as a 'query object'.

The reference object or the query object may be a spot existing in a specific area. For example, reference objects or query objects may include the Eiffel Tower, the Statue of Liberty, Bulguksa Temple, and so on.

The representative feature generation unit 120 may extract a predetermined point from the reference object included in the reference image. The extracted predetermined point is an environment-insensitive point such as enlargement, reduction or image rotation of the image. For example, the point extracted in the SURF (Speeded Up Robust Features) algorithm may be the predetermined point.

The SURF algorithm is one of the algorithms to find a feature that is invariant to the environment change considering the environmental changes such as scale, illumination, and viewpoint from multiple images. It is generally compared with SIFT It is an algorithm that shows a performance and greatly improves the speed.

The representative characteristic generation unit 120 may extract (or generate) one representative characteristic by merging a plurality of predetermined points in consideration of the object type. The predetermined point may be a representative point extracted from each of the reference images. Thus merging a plurality of points may include merging a plurality of points each extracted from a plurality of images into one feature.

The reason why the plurality of predetermined points are merged and generated as one representative feature is that the user can quickly search for features matching the query image when the query image is received from the user terminal 200. [

The indexing unit 130 may index the generated or extracted representative features. For example, the indexing unit 130 may index the generated or extracted representative features with the name of the object (e.g. name of attraction).

The database 140 may store various information or data. For example, the database 140 may store a reference image including a reference object, a representative characteristic of the reference object, specific information mapped to the representative characteristic (e.g., detailed information about a specific attraction), and the like.

When the comparison unit 150 receives the query image from the user terminal 200, it may compare the similarity between the query object and the reference object stored in the database 140. For example, the comparison unit 150 may compare a representative point extracted from the query object (e.g., extracted by the SURF algorithm) with a representative feature stored in the database 140 ) Can be compared with each other. The comparing unit 150 can extract a representative feature that satisfies the degree of similarity higher than the specific reference. And extract information mapped to the representative feature. For example, when the extracted representative feature is indexed with an Eiffel tower, the comparison unit 150 may extract information mapped to the Eiffel tower.

A similarity comparison between the representative points extracted from the query object and the representative features stored in the database 140 may be performed using Equation (1).

(D is the merged SURF descriptor, D (i) is the ith component of D, d is the regular SURF descriptor, and d (i) is the ith component of d.

The transmitting unit 160 may transmit information (for example, detailed information of a specific landmark) to the user terminal 200 based on the comparison result of the comparing unit 150.

The control unit 170 controls the receiving unit 110, the representative tone generating unit 130, the indexing unit 130, the database 140, the comparing unit 150 and the transmitting unit 160 as a whole can do.

3 is a flowchart showing an information providing method according to an embodiment of the present invention.

The information providing apparatus 100 may receive a reference image corresponding to a specific query from the web server 300 and may collect information mapped to the reference object included in the reference image (S310).

For example, the step S310 as the information collecting step may be performed as shown in FIG.

The receiving unit 110 transmits a local information query including the latitude and longitude to the web server 300 and transmits a reference image corresponding to the local information query from the web server 300 May be received (S311). Wherein the information about the first reference image includes at least one of the first reference image, the metadata for the first reference image, the ID of the first reference image provider, the URL information for identifying the first reference image, The title of the image, and the like.

The control unit 170 may extract a specific keyword query in consideration of the frequency of exposure in the information on the received first reference image (S312). For example, a word with the highest exposure in the title of the first reference image can be extracted by a specific keyword. The extracted word may be a single word or a plurality of words may be continuous phrases.

The transmitting unit 160 may transmit the extracted specific keyword query to the web server 300. The receiving unit 110 may receive information on a reference image corresponding to the specific keyword query (for convenience, referred to as a 'second reference image') (S313). In this case, the receiving unit 110 may also receive a search result (e.g., detailed information on a specific keyword) for the specific keyword query. Wherein the information about the second reference image includes at least one of the second reference image, the metadata for the second reference image, the ID of the second reference image provider, the URL information for identifying the second reference image, The title of the image, and the like.

The representative feature generation unit 120 may extract or generate a representative feature from the reference object included in the second reference image (S320). For example, in order to recognize a specific object in real time, the representative characteristic generation unit 120 can automatically find a part of the object, which is considered to be a feature of the object, from the training images of the object. And it can be determined which point of interest is a representative part of the object among various points of interest (for example, points extracted through the SURF algorithm). For example, if all points of interest are compared with each other, and the similarity exceeds a certain value, that part is selected as a representative feature. That is, one representative feature can be extracted (or generated) by merging a plurality of points of interest in consideration of the object type. The point of interest may be a representative point extracted from each reference image. Thus merging a plurality of points may include merging a plurality of points each extracted from a plurality of images into one feature.

The indexing unit 130 may appropriately index the set of representative features that resulted from step S320 and effectively process the query when a subsequent query is received from the user terminal 200. [ Since the result of the GPS sensor is composed of the latitude and the longitude, the indexing unit 130 can consider the entire region of the earth as a square box of -180 to 180 degrees of the radius of the earth and construct an r-tree for this region have.

The control unit 170 may maintain the database 140 in which the reference image including the reference object, the representative characteristic of the reference object, the specific information mapped to the representative characteristic (e.g., detailed information about a specific attraction) (S340).

The receiving unit 110 may receive a query image for requesting specific information from the user terminal 200 (S350). In this case, the receiving unit 110 may receive local information corresponding to the query image. For example, if the query image is photographed in a particular area, information about latitude and longitude of the photographed area may be received with the query image.

The comparing unit 150 may compare the similarity between the query object of the query object included in the query image and the reference object stored in the database 140 (S360). The comparing unit 150 can extract a representative feature that satisfies the degree of similarity higher than the specific reference. The similarity comparison may be performed using Equation (1). In this case, the comparison unit 150 may extract representative features by considering the received local information together with the query image. For example, when there are three representative features (representative feature 1, representative feature 2, and representative feature 3), only representative features (e.g., representative feature 1) mapped to the received local information among the three representative features Can be extracted. The comparing unit 150 may extract information mapped to the extracted representative feature.

The transmitting unit 160 may transmit the information mapped to the representative characteristic to the user terminal 200.

FIG. 5 shows an example in which information mapped to the representative feature is transmitted to the user terminal 200. FIG. 5 (a) shows an example of a query image to be transmitted from the user terminal 200 to the information providing apparatus 100, and FIG. 5 (b) shows an example of a query image to be transmitted to the information providing apparatus 100 to the user terminal 200. In the example of FIG.

As described above, in the information providing method according to an embodiment of the present invention, a plurality of points extracted from a reference object are merged and generated as one representative feature, so that a representative feature matched with a representative point extracted from a query object is quickly retrieved can do.

The above-described information providing method may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable recording medium. At this time, the computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. On the other hand, the program instructions recorded on the recording medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.

The computer-readable recording medium includes a magnetic recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical medium such as a CD-ROM and a DVD, a magnetic disk such as a floppy disk, A magneto-optical media, and a hardware device specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

The recording medium may be a transmission medium such as a light or metal line, a wave guide, or the like including a carrier wave for transmitting a signal designating a program command, a data structure, and the like.

The program instructions also include machine language code, such as those generated by the compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The above-described information providing method and information providing apparatus are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified so that all or some of the embodiments are selectively As shown in FIG.

100: Information providing device
110:
120: representative characteristic generating unit
130: indexing unit
140: Database
150:
160:
170:
200: user terminal
300: Web server

Claims (5)

A method of providing an information providing apparatus with information corresponding to an image,
The information providing apparatus collecting the specific image and the information about the specific image;
The information providing apparatus merging at least two points of the reference object included in the collected specific image to generate one representative feature;
The information providing device indexing the generated representative feature;
Receiving the query image from the user terminal;
Comparing the representative feature with a predetermined point of the query object included in the received query image; And
And transmitting the information mapped to the query image to the user terminal based on the comparison result. The method according to claim 1,
Wherein the query image receiving step includes receiving local information corresponding to the query image,
Wherein the information mapped to the query image is information on which the received local information is considered. 2. The method according to claim 1,
Receiving an image corresponding to a specific local information query;
Extracting a specific keyword by considering the frequency of the information on the received image; And
And collecting an image corresponding to the keyword query corresponding to the extracted specific keyword query. The method according to claim 1,
Wherein the predetermined feature point and the representative feature comparison step are performed using Equation (1). ≪ EMI ID = 1.0 >
[Equation 1]

(D is the merged SURF descriptor, D (i) is the ith component of D, d is the regular SURF descriptor, and d (i) is the ith component of d. 2. The method according to claim 1,
And merging at least two points of the reference object in consideration of the type of the reference object.

Images