KR101017016B1 - Method, system and computer-readable recording medium for providing information on goods based on image matching - Google Patents

Abstract

The present invention relates to a method, a system and a computer readable recording medium for providing commodity information using image matching. According to an aspect of the present invention, a method for providing information regarding a product related to an input image, wherein a plurality of product identification information is input to an image search interface, the image data output for each of the product identification information is inputted. Generating at least one model group in association with the information, storing information about the generated model group in a model database, obtaining a search image using input information from a user terminal device, and using the search image Determining at least one model group, and providing product information to the user terminal device using product identification information associated with the determined model group. According to the present invention, when an image related to a specific product is input, a new product information providing model for providing information related to the specific product can be achieved based on a plurality of image data accumulated by users.

Image Search, Product Search, Shopping

Description

Method of providing product information based on image matching, system and computer readable recording medium {METHOD, SYSTEM AND COMPUTER-READABLE RECORDING MEDIUM FOR PROVIDING INFORMATION ON GOODS BASED ON IMAGE MATCHING}

The present invention relates to a method, a system and a computer readable recording medium for providing commodity information based on image matching. More specifically, the present invention receives information about the image of the search target product from the user terminal device, retrieves a comparison target image similar to the received image, and then uses the product identification information obtained from the comparison target image A method, a system and a computer readable recording medium for providing product information to a customer.

In recent years, as the use of the Internet has become more common, users can obtain various information through Internet searches. That is, after accessing an Internet search site through a terminal device such as a personal computer that can access the Internet, users can search for various contents related to news, knowledge, games, communities, and the like. Such a search site mainly provides a search interface based on text data. More specifically, the search site is implemented by receiving a search keyword from the user and providing the user with a list of search results corresponding to the input search keyword.

On the other hand, with the development of communication technology and the increase of the storage capacity, the ratio of documents including image data such as photographs and videos as well as documents in the form of text is rapidly increased on the Internet. Due to the influence of culture, the rate of increase is intensifying.

However, despite the situation where a huge amount of image data is accumulated on the Internet, image search interfaces provided by search sites have not yet escaped from the category of text search interfaces. For example, the image search function currently provided by most search sites receives a search keyword in text form from a user, an image including the input search keyword as tag information, or a web document including the input search keyword. It's just a level of providing the included images as search results. Therefore, under the current situation, a user who wants to obtain information about a product related to image data can perform a search for the corresponding product by inputting various clues that the user can visually obtain from the image data in a text format. There was only.

In order to solve this problem, a technology for recognizing a product based on a shape recognition technology and providing information on the recognized product has been developed. More specifically, first, after constructing a sample database of a plurality of products, when image data is input from a user, an area in which the product exists or an area in which the product is expected to exist is detected and detected in the image data. This is a method of recognizing the goods displayed in the corresponding area by comparing the image of the product area with the sample data stored in the sample database.

However, according to the above conventional technology, while most of the products have a three-dimensional structure, the appearance of the products obtained from the image data is only flat information, so that the product information providing technology based on the conventional image matching technology In order to be practical, a large amount of sample images photographed from various angles was required for each product.

In some cases, despite the image data associated with a particular product, the product may not appear in the image data itself. For example, a scene in which the specific product is not included in the CF is engraved on the public. Thus, it may be assumed that the public will see the scene and think of a specific product. However, in such a case, there is a disadvantage in that it is difficult to obtain an appropriate result if the conventional technique of recognizing a product based on the shape recognition technique as described above is used.

It is an object of the present invention to solve all the problems described above.

In addition, an object of the present invention is to analyze a plurality of image data accumulated by the users to maintain the image model data of the product, and to recognize the product through the image matching operation with the retained image model data.

In addition, an object of the present invention is to recognize a product associated with the image data with high accuracy while omitting the process of detecting the product region from the image data.

In addition, an object of the present invention is to extract the identification information of the product from the tag information provided in the image data, and to provide the product information through the extracted product identification information.

Representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, a method for providing information regarding a product related to an input image, wherein a plurality of product identification information is input to an image search interface, the image data output for each of the product identification information is inputted. Generating at least one model group in association with the information, storing information about the generated model group in a model database, obtaining a search image using input information from a user terminal device, and using the search image Determining at least one model group, and providing product information to the user terminal device using product identification information associated with the determined model group.

According to another aspect of the present invention, there is provided a method for providing information regarding a product related to an input image, the method comprising: acquiring a search image using input information from a user terminal device; image data stored in the image database; Calculating similarity with each other, determining at least one image data based on the calculated similarity, obtaining at least one tag information associated with the determined image data, and using the acquired tag information. Obtaining product identification information, and providing product information to the user terminal device using the obtained product identification information.

According to still another aspect of the present invention, there is provided a system for providing information regarding a product related to an input image, wherein when a plurality of product identification information is input to an image search interface, the image data outputted by the product identification information is output. A model data management unit generating at least one model group in association with identification information, a model database storing information on the model group generated by the model data management unit, and obtaining a search image using input information from a user terminal device; An image obtaining unit, an image matching unit to determine at least one model group using the search image, and a product information providing unit to provide product information to the user terminal device using product identification information associated with the determined model group. Provided with the included system .

According to still another aspect of the present invention, there is provided a system for providing information regarding a product related to an input image, the system comprising: an image obtaining unit for obtaining a search image by using input information from a user terminal device; Comparing the stored image data to calculate the similarity, and obtains an image matching unit for determining at least one image data based on the calculated similarity, and at least one tag information associated with the determined image data, the acquisition And a product information providing unit for acquiring product identification information using the acquired tag information and providing the product information to the user terminal device using the obtained product identification information.

In addition, there is provided another method, system, and computer readable recording medium for recording a computer program for carrying out the method for implementing the present invention.

According to the present invention, when an image related to a specific product is input, a new product information providing model for providing information related to the specific product can be achieved based on a plurality of image data accumulated by users.

According to the present invention, in the process of recognizing the product related to the image data, the process of detecting the product region is omitted, thereby achieving an effect that can simplify the product recognition process.

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

DETAILED DESCRIPTION Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

[Preferred Embodiments of the Invention]

Configuration of the entire system

1 is a diagram schematically illustrating a configuration of an entire system for determining product related thereto and providing product information when image data is input to a product information providing system according to an exemplary embodiment of the present invention.

As shown in FIG. 1, the entire system according to an embodiment of the present invention includes a communication network 100, a product information providing system 200 that recognizes a product related to image data, and provides product information, and a plurality of images. The image server 300 may store data, the user terminal device 400, and the shopping server 500.

First, the communication network 100 may be configured regardless of its communication mode such as wired and wireless, and may include a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Network). Preferably, the communication network 100 according to the present invention may be a known World Wide Web (WWW).

According to an embodiment of the present invention, the product information providing system 200 receives image data from the user terminal device 400 or the image server 300, the user terminal device to receive information about the product related to the received image data 400 may be provided.

In addition, according to an embodiment of the present invention, the product information providing system 200 classifies the images stored in the at least one image server 300 according to a predetermined rule to generate a product model data group. By using the product model data, it is possible to recognize a product related to image data received from the user terminal device 400 or the image server 300.

In addition, according to an embodiment of the present invention, the product information providing system 200 may perform a function of allowing the user terminal device 400 to be connected to the shopping server 500 so that the user can purchase the recognized product. have.

According to an embodiment of the present invention, the image server 300 includes a function of storing information regarding a plurality of image data and providing image data to the user terminal device 400 or the product information providing system 200. It can be a server. Specifically, the image data stored in the image server 300 may be image data generated or stored by a user of the user terminal device 400 or an administrator of the image server 300.

According to an embodiment of the present invention, the image server 300 may include a predetermined search interface and, when a search condition is input, may output information about image data corresponding to the input search condition. For example, when "pradaphone" is input as a search condition, image data including "Pradaphone" as tag information or information about various contents related to "Pradaphone" may be output.

According to an embodiment of the present invention, the image server 300 may be an operation server of an internet portal site, a community site, a shopping site, or a product review site.

Meanwhile, although the image server 300 is illustrated as being separately configured from the product information providing system 200 and the shopping server 500 in FIG. 1, the image server 300 may be used according to the needs of those skilled in the art for implementing the present invention. May be included in the product information providing system 200 or the shopping server 500.

The user terminal device 400 according to an embodiment of the present invention has a function of enabling communication after connecting to a remote computer such as a product information providing system 200, an image server 300, and a shopping server 500. As a digital device including a personal computer (for example, a desktop computer, a laptop computer, etc.), a workstation such as a workstation, a PDA, a web pad, a mobile telephone, etc. Any number may be adopted as the user terminal device 400 according to the present invention.

According to an embodiment of the present invention, the shopping server 500 performs communication with the product information providing system 200 and / or the user terminal device 400. The shopping server 500 receives the product identification information from the product information providing system 200 or the user terminal device 400, and provides an interface for purchasing a product corresponding to the received product identification information. Can be provided to In addition, although the product information providing system 200 and the shopping server 500 are illustrated separately in FIG. 1, according to the needs of those skilled in the art to implement the present invention, the shopping server 500 may provide a product information providing system. It may be included in the 200.

Composition of Product Information Provision System

Hereinafter, the internal structure of the product information providing system 200 according to the present invention and the function of each component will be described.

2 is a diagram illustrating in detail the internal configuration of the product information providing system 200 according to an embodiment of the present invention. 2, the product information providing system 200 according to an exemplary embodiment of the present invention may include an image obtaining unit 210, a model data managing unit 220, an image matching unit 230, and an associated keyword providing unit 240. ), A product information providing unit 250, a database manager 260, a communication unit 270, and a controller 280. According to an exemplary embodiment of the present invention, the image acquisition unit 210, the model data management unit 220, the image matching unit 230, the associated keyword providing unit 240, product information providing unit 250, database management unit 260 The communication unit 270 and the control unit 280 may be program modules in which at least some of them communicate with the image server 300, the user terminal device 400, and / or the shopping server 500. Such program modules may be included in the product information providing system 200 in the form of an operating system, an application module, and other program modules, and may be physically stored on various known storage devices. In addition, such program modules may be stored in a remote storage device that can communicate with the product information providing system 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

First, the image acquisition unit 210 according to an embodiment of the present invention receives image data from the user terminal device 400 or obtains identification information on the image data from the user terminal device 400, and uses the same. Image data may be obtained from the image server 300.

Next, the model data management unit 220 according to an embodiment of the present invention groups the image data stored in the image server 300 according to a predetermined criterion, and the grouped image data and the identification information of the product (for example, , A product name, a model name, etc.) may be stored in the model database 260a to be described later.

The image matching unit 230 according to an embodiment of the present invention calculates the similarity between the image acquired by the image acquisition unit 210 and the image data stored in the model database 260a and / or the image server 300. Based on the calculated similarity, the model group or image data having the highest n similarities may be determined.

The associated keyword providing unit 240 according to an embodiment of the present invention obtains the associated keyword information from the associated keyword database 260b to be described later, and transmits the related keyword information to the model data management unit 220 and / or the product information providing unit 250. You can perform the function provided.

The product information providing unit 250 according to an exemplary embodiment of the present invention obtains identification information of a product related to the model group or the image data determined by the image matching unit 230. In this process, the product information providing unit 250c may refer to the product information database 260c which will be described later. The product information provider 250 may provide the product information to the user terminal device 400 using the obtained product identification information. In addition, the product information providing unit 250 may provide a function of allowing the user to be connected to a purchase page for the corresponding product by providing the acquired product identification information to the shopping server 500.

The database manager 260 according to an embodiment of the present invention includes a model database 260a in which model data to be compared is stored, an associated keyword database 260b in which information on related keywords are stored, and a product in which product information is stored. Information database 260c.

In the communication unit 270 according to an embodiment of the present invention, the product information providing system 200 according to the present invention communicates with an external device such as an image server 300, a user terminal device 400, and a shopping server 500. It can do a function that makes it possible.

According to an embodiment of the present invention, the controller 280 may include an image receiver 210, a model data manager 220, an image matcher 230, an associated keyword provider 240, a product information provider 250, and a database. It performs a function of controlling the flow of data between the manager 260 and the communication unit 270.

In the above description, the database storing information for the implementation of the present invention is classified into three databases, a model database 260a, an associated keyword database 260b, and a product information database 260c. Of course, can be changed according to the needs of those skilled in the art.

Finally, in the present specification, a database is a concept that includes not only a negotiated database but also a broad database including a data file based on a computer file system, and the like. It should be understood that if data can be extracted, it can be included in the database referred to in the present invention.

Generation of model data

According to an embodiment of the present invention, the model data management unit 220 groups the image data stored in the image server 300 according to a predetermined criterion, and associates the grouped image data with the identification information of the product to obtain a model database ( 260a).

According to an embodiment of the present invention, the image data stored in the image server 300 may be image data generated or stored by a user, a content provider, or an administrator of the image server 300. In addition, the image data in the present invention does not necessarily mean only the image data itself, but should be understood as a concept that includes all types of data including the image data. For example, the image server 300 may include image data directly uploaded by a user or an administrator of the image server 300, an image attached to a post created in a personal blog, an image attached to a news article provided by a news provider, a product, and a product. Information about an image attached to a review material contributed to a review site may be stored.

According to an embodiment of the present invention, the image data stored in the image server 300 may include tag information related to the image.

Here, the tag information refers to data in text format stored in association with the data in order to indicate an attribute of the data, and the tag information is designated by at least one of a user, a content provider, and an administrator of the image server 300. If the image server 300 supports a predetermined automatic tagging process, it may be automatically given by the image server 300. For example, if a user writes a review data about a mobile phone, the user who created the review data or an administrator of the review site may assign the product name of the mobile phone as tag information of the review data, and the image server 300 automatically The tagging program may analyze the contents of the review data and automatically assign tag information that frequently appears in the review data, such as a product name of a mobile phone. According to an embodiment of the present invention, the automatic tagging program may be included in a known search engine.

According to an embodiment of the present invention, the model data manager 220 transmits a search condition to the image server 300 periodically or aperiodically, and provides information about the image data provided as a response to the model database 260a. Can be stored in At this time, the search condition transmitted from the model data management unit 220 may be a product name stored in the product information database 260c, but is not necessarily limited thereto, and may function as the search condition if an appropriate search result is obtained. .

For example, when the model data management unit 220 inputs "pradaphone" to the image server 300 as a search condition, the image server 300 displays image data including "pradaphone" in the tag information. Provided to the management unit 220. The model data manager 220 may generate a model group by linking the information about the image data provided from the image server 300 and the search condition transmitted to the image server 300 and store the model group in the model database 260a. .

The model data management unit 220 may associate a model group composed of a plurality of image data for each product stored in the product information database 260c based on the image search operation performed as described above.

According to an embodiment of the present disclosure, the model data manager 220 may integrate image data for a plurality of search conditions into one group with reference to the associated keyword database 260b. For example, if "pradaphone", "LG-SB310", and "LG-LB3100" are designated as the related keywords in the related keyword database 260b, a group of the respective search results for them are combined and grouped together. Can be created with In this case, the search condition stored in association with the integrated image data may include all of “pradaphone”, “LG-SB310”, and “LG-LB3100”, or may include only some of them.

3 is a diagram exemplarily illustrating data stored in the model database 260a.

Referring to FIG. 3, it can be seen that a plurality of image data are linked to one model group. In the example of FIG. 3, the information stored in the image data field may be image data itself provided from the image server 300 or path information for acquiring image data.

Acquisition of Input Image

According to an embodiment of the present disclosure, the image acquisition unit 210 may receive image data from the user terminal device 400. For example, when a user accesses the product information providing system 200 and inputs an image related to a product he / she wants to search as a query, the image obtaining unit 210 may receive the input image.

According to another exemplary embodiment of the present disclosure, the image acquisition unit 210 may receive identification information about image data from the user terminal device 400. For example, when a user clicks on specific image data included in the web document with a mouse, the identification information of the image data may be transmitted to the image acquisition unit 210. The image acquisition unit 210 may acquire the image data in the product information providing system 200 using the image data identification information, or may access the external system and receive the image data. Here, the image data identification information may be a concept including path information for acquiring image data.

Recognition of goods

According to the first embodiment of the present invention, the image matching unit 230 compares the input image acquired by the image obtaining unit 210 with respect to each model group stored in the model database 260a, and inputs the same. Similarity can be calculated between the image and the model data.

Techniques for determining the similarity between images are published by Clark F. Olson, "Maximum-Likelihood Image Matching," a paper published in "IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MATCHING INTELLIGENCE", Vol. 24, No. 6 (June 2002), and At least one of the well-known techniques, such as "Video Local Pattern Based Image Matching for Visual Mapping," which is written by Lei Wang and two others and described in Volume 3 of the 18th International Conference on Pattern Recognition. (The content of the above articles should be considered to be incorporated in its entirety.) However, the present invention is not limited to and interpreted by the well-known technologies recited above. To reveal.

According to an embodiment of the present invention, the image matching unit 230 may determine at least one model group based on the calculated similarity.

The product information providing unit 250 according to an embodiment of the present invention obtains the identification information of the product corresponding to the model group determined by the image matching unit 230, and then stores information about the corresponding product in the user terminal device ( 400). The information on the corresponding product may include a name of the product, a price of the product, information of a merchandise vendor such as a site or a web document where the product can be purchased, and the like. In addition, the product information providing unit 250 may perform a function to allow the user terminal device 400 to be automatically connected to the purchase page for the product.

According to the second embodiment of the present invention, the image matching unit 230 may recognize the product using the image data stored in the image server 300 without using the model data stored in the model database 260a.

More specifically, the image matching unit 230 compares the image acquired by the image acquisition unit 210 with the image data stored in the image server 300 to calculate the similarity, and based on the calculated similarity at least one Image data can be determined.

After obtaining the tag information of the image data determined by the image matching unit 230, the product information providing unit 250 according to an embodiment of the present invention to provide information about the corresponding product to the user terminal device 400. Can be. The information on the corresponding product may include a name of the product, a price of the product, information of a merchandise vendor such as a site or a web document where the product can be purchased, and the like. In addition, the product information providing unit 250 may perform a function to allow the user terminal device 400 to be automatically connected to the purchase page for the product.

In the second embodiment of the present invention, product identification information is obtained using tag information associated with image data stored in the image server 300. The tag information associated with the image data is a user who generates or stores image data. , Information provided by the content provider, etc., may include information that is not related to the identification information of the product. For example, when a user creates a post containing a photo of a newly purchased mobile phone, the tag information includes information that is irrelevant to information that identifies the product name of the mobile phone such as "birthday gift" and "treasure number 1". We can assume the case of input. Therefore, in order to prevent such a case, the product information providing unit 250 according to an embodiment of the present invention compares the obtained tag information with the product name information stored in the product information database 260c, and the matching data is matched. Only if there is a modification can be assumed, such as considering the acquired tag information as product identification information.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

1 is a diagram schematically illustrating a configuration of an entire system for recognizing a product related to image data and providing product information according to an embodiment of the present invention.

2 is a diagram illustrating in detail the internal configuration of the product information providing system 200 according to an embodiment of the present invention.

3 is a diagram exemplarily illustrating data stored in the model database 260a.

<Brief description of the major reference numerals>

100: network

200: product information providing system

210: image acquisition unit

220: model data management unit

230: the image matching unit

240: related keyword provider

250: product information provider

260: database management unit

260a: model database

260b: Association keyword database

260c: product information database

270: communication unit

280: control unit

300: image server

400: user terminal device

500: shopping server

Claims (25)

A method performed in a system for providing information on a product related to an input image in association with an image server, When a plurality of product identification information for product search is input to an image search interface, at least one image data of a product selectively obtained through a search based on the product identification information among image data stored in the image server, according to at least one product model. Classifying into a model group to generate linkage information between a model group, product identification information, and image data in association with the product identification information, and storing and storing the generated linkage information in a model database; Obtaining a search image related to a product to be searched by a user using input information from a user terminal device; The at least one model group is determined by comparing the acquired search image with image data classified in at least one model group which is selectively obtained through a search based on the product identification information and accumulated and stored in the model database. Steps, and Providing product information to the user terminal device using product identification information associated with the determined model group How to include. The method of claim 1, And the image retrieval interface outputs image data stored corresponding to the input product identification information. The method of claim 2, The image data is stored in a predetermined image server in association with tag information indicating an attribute of the image data, And the image search interface outputs image data including the input product identification information as tag information. The method of claim 3, The tag information is given by the creator of the image data or the administrator of the image server, Method automatically assigned by a predetermined tagging program. The method of claim 4, wherein And the tagging program tags the image data using text data included in a web document including the image data. The method of claim 1, Acquiring the search image, Is performed by receiving the search image from the terminal device, or And receiving search image identification information from the terminal device, and obtaining the search image by using the received search image identification information. The method of claim 1, Determining the model group, Calculating a similarity by comparing the searched image with image data included in at least one model group stored in the model database; Obtaining image data corresponding to the top n of high similarity, and Selecting at least one of a model group corresponding to the n image data Method comprising a. The method of claim 1, Wherein the product information includes at least one of a name, a price, and information about a seller of the product. The method of claim 8, Providing the product information, And controlling the user terminal device to access a site or a web document from which the product can be purchased. The method of claim 1, Storing in the model database, Incorporating model groups corresponding to the product identification information associated with each other into a single model group. delete delete A system for providing information on a product related to an input image in association with an image server, When a plurality of product identification information for product search is input to the image search interface, image data of a product selectively obtained by searching based on the product identification information among the image data stored in the image server may be at least one model for each product model. A model data management unit classifying into groups and generating linkage information between a model group, product identification information, and image data in association with the product identification information; A model database in which linkage information between the model group, product identification information, and image data generated by the model data management unit is accumulated and stored; An image acquisition unit for obtaining a search image related to a product to be searched by a user using input information from a user terminal device; The at least one model group is determined by comparing the acquired search image with image data classified in at least one model group which is selectively obtained through a search based on the product identification information and accumulated and stored in the model database. An image matching unit, and Product information providing unit for providing product information to the user terminal device using the product identification information associated with the determined model group System comprising a. The method of claim 13, And the image search interface outputs stored image data corresponding to the input product identification information. The method of claim 14, The image data is stored in a predetermined image server in association with tag information indicating an attribute of the image data, And the image search interface outputs image data including the input product identification information as tag information. The method of claim 15, The tag information is given by the creator of the image data or the administrator of the image server, A system characterized by being automatically assigned by a predetermined tagging program. The method of claim 16, The tagging program is a system for tagging the image data using the text data included in the web document containing the image data. The method of claim 13, The image acquisition unit, Receiving the search image from the terminal device, or And receive search image identification information from the terminal device, and obtain the search image by using the received search image identification information. The method of claim 13, The image matching unit, Comparing the searched image with the image data included in at least one model group stored in the model database to calculate a similarity, obtain image data corresponding to the top n high similarity, and the n image data And select at least one of the group of models corresponding to. The method of claim 13, And the product information includes at least one of a name of the product, a price, and information about a seller of the product. 21. The method of claim 20, The product information providing unit, And controlling the user terminal device to access a site or a web document from which the product can be purchased. The method of claim 13, The system maintains a list of associated product identification information, The model data management unit, And model groups corresponding to product identification information associated with each other by referring to the list into one model group. delete delete A computer-readable recording medium for recording a computer program for executing the method according to any one of claims 1 to 10.

Images