KR20120052569A - Method for providing image management based on user information and device and system using the same - Google Patents

Abstract

PURPOSE: A user information base image search management method and an apparatus and system using the same are provided to store an image recorded by a user in the image object database and to utilize the image for image search of other user. CONSTITUTION: An image photographing mean(202) takes a photograph of an image of a product. An input means(204) inputs product information related to the product. An information management means(203) classifies and stores the image of the product and the product information. A transceiver(201) transmits the image of the product and the product information to an image information managing apparatus through a communications network. The transceiver transmits a request in regard to product information search and receives the product information which is searched from the image information managing apparatus. An information search means(205) searches the product information saved in the information management means.

Description

User Information-based Image Retrieval Management Method and Device and System Using the Method {METHOD FOR PROVIDING IMAGE MANAGEMENT

The present invention relates to an image retrieval management method for retrieving an input image during mobile search or web search and to managing a database, and an apparatus using the same. More specifically, the present invention relates to a method for diversifying a search method and increasing search efficiency by using an image and information database (DB, database) registered by other users when searching for an input image. It uses search technology, graphic processing technology, object database generation technology, wired and wireless network technology.

Conventional image recognition and retrieval techniques involve searching for corresponding information by detecting feature information from a video image and comparing the detected value with a stored database. Recently, due to the proliferation of smartphones and web cams equipped with high-performance cameras and the development of camera technology and image processing technology, research on image retrieval methods is gradually being activated.

In the conventional image retrieval method, the image is taken using a camera such as a PC camera or a mobile phone, and a feature point is extracted from an edge or grid obtained from the obtained image and compared with a pre-registered feature information database. Through video recognition and related information is retrieved.

However, in the conventional image retrieval technology, even if the image of the same object is extracted due to the change of illumination, the change of angle, the rotation of the object, and the background of the object, different feature information is extracted and image recognition is not performed properly. There are many. In addition, there is a problem in that the image search results are different for the same object due to the difference between the imaging tool and the photographer. In addition, since the work of continuously updating and databaseging newly generated images is very large, an integrated solution for searching all images is not completed.

The present invention has been made to solve the above problems, a method and apparatus that can increase the search efficiency by the database of information by the user input through the construction of the image object database by the user participation and diversified the search method And related systems.

To this end, according to the first aspect of the present invention, the image retrieval management system according to the present invention, photographing the image of the product and stores the image of the product and product information associated with the product, and the image of the product and the product information A plurality of user terminals capable of transmitting the information through a communication network and searching the product information stored therein; And classify and store the image of the product and the product information received from the plurality of user terminals, and responsive to the received image search request from one user terminal among the plurality of user terminals, the image of the product and the product information. It includes an information management device to search for and provide to the one user terminal.

According to a second aspect of the present invention, a user terminal according to the present invention comprises: image capturing means for capturing an image of a product; Input means for inputting product information related to the product; Information management means for classifying and storing the image of the product and the product information; And transmitting the image of the product and the product information to a video information management device through a communication network, transmitting a request for product information retrieval using the video information management device, and receiving the searched product information from the video information management device. It includes a transceiver for.

According to a third aspect of the present invention, an information management apparatus connected to a plurality of user terminals and a communication network according to the present invention comprises: a transceiver for communicating with the plurality of user terminals; A feature information extractor which generates an image thumbnail from images of goods received from the plurality of user terminals, extracts feature information from the image thumbnail, classifies the image of the goods and product information related to the goods An image information management device including a product information management unit to store the information; And a shopping information manager searching for shopping information related to the product.

According to a fourth aspect of the present invention, an image management method used in a system in which a plurality of user terminals and an information management apparatus is connected through a communication network according to the present invention comprises: capturing an image of a product and related to the image of the product and the product Storing product information; Transmitting an image of the product and the product information to the information management device; And classifying and storing the image of the product and the product information received from the plurality of user terminals, and generating shopping information related to the product.

According to a fifth aspect of the present invention, an image management method used in a user terminal connected to an information management apparatus through a communication network according to the present invention comprises: photographing an image of a product and displaying the image of the product and product information related to the product; Storing in the user terminal; Generating an image thumbnail from the image of the product and extracting feature information from the image thumbnail; Classifying and storing the image of the product and the product information by using the feature information in the user terminal; And transmitting the image of the product and the product information to the information management device.

According to a sixth aspect of the present invention, an image management method used in an information management apparatus connected to a plurality of user terminals and a communication network according to the present invention comprises: receiving a video of a product and product information related to the product from the plurality of user terminals; Receiving; Generating an image thumbnail from images of goods received from the plurality of user terminals and extracting feature information from the image thumbnail; And classifying and storing the image of the product and the product information by using the feature information.

According to the present invention, it is possible to provide an active search method by user participation rather than the conventional manual search method, and to use the image object database to photograph the image taken by the user and register it in the server, and then search the image of other users as well as themselves. .

In addition, according to the present invention, there is an effect of increasing the search efficiency by the database of various images and information by the user participation and by diversifying the search method.

In addition, according to the present invention, by sharing the data by the user participation, it is possible to reduce the cost required to build the image object database, and to provide a convenient and various search methods and user-centered interface to the user when searching for goods. .

1 is a conceptual diagram showing the configuration of an image search management system according to an embodiment of the present invention.
2 is a conceptual diagram showing the configuration of a user terminal according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a configuration of an information management apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating an image search management method performed in an image search management system according to an embodiment of the present invention.
5 is a flowchart illustrating an image search management method performed in a user terminal according to an embodiment of the present invention.
6 is a flowchart illustrating a video retrieval management method performed in an information management apparatus according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and the operation and effect thereof will be clearly understood through the following detailed description.

Prior to the detailed description of the present invention, the same components will be denoted by the same reference numerals even if they are displayed on different drawings, and the detailed description will be omitted when it is determined that the well-known configuration may obscure the gist of the present invention. do.

1 is a conceptual diagram illustrating a configuration of an image search management system according to an embodiment of the present invention. Referring to FIG. 1, an image search management system includes a plurality of user terminals 100, a communication network 200, and an information management apparatus 300.

The plurality of user terminals 100 may include various types of user terminals capable of transmitting and receiving data and capturing images through the communication network 200. For example, the plurality of user terminals 100 include various mobile communication terminals including smart phones, PCs, PDAs, digital cameras, web cams, fixed or mobile subscriber units, or any other type of device operable in a wired or wireless environment. . The plurality of user terminals 100 may communicate with each other through the communication network 200, and the plurality of user terminals 100 may communicate with the information management apparatus 300 through the communication network 200.

The communication network 200 includes various networks that provide wired and wireless communication to the plurality of user terminals 100 and the information management apparatus 300. For example, the communication network 200 may be a wired internet network, and may include a wireless data network connected through a mobile communication network (CDMA, WCDMA, WiBro, etc.), or an internet network connected through short-range communication. Specifically, the CDMA network may be connected to a data network through a PDSN (Packet Data Serving Node), and the W-CDMA network may be connected to a data network through a Gateway GPRS Support Node (GGSN). In addition, in an area such as a hot-spot that is accessible to an access point (AP), the Internet network may be connected through short-range communication such as Bluetooth or Wi-Fi.

Such an internet network is a network existing at the TCP / IP protocol and its upper layer, and includes HTTP (Hypertext Transfer Protocol), Telnet, File Transfer Protocol (FTP), Domain Name Server (DNS), Simple Mail Transfer Protocol (SMTP), Simple Network Management Protocol (SNMP), Network File Service (NFS), and Network Information Service (NIS).

The plurality of user terminals 100 may photograph an image of a product by using a camera or a web cam mounted on the user terminal, respectively. In addition, not only the captured image may be stored but also product information related to the product may be input together with the image of the product. For example, the user may further store user input information about a product together with an image of the product using the user terminal 100. The user input information may include, for example, a product name, price information, a manufacturer, a release date, a purchase review, and the like.

Meanwhile, the merchandise image mentioned below may be a photographic image of a merchandise directly photographed using the user terminal 100, but is not limited thereto and may be an image obtained through a web site or an image obtained through another terminal. .

The plurality of user terminals 100 may store product images and product information, respectively, in the terminal. For example, the plurality of user terminals 100 may store the product image and the product information in the form of a cache, so that when the image search is performed, the product image and the product information stored in the terminal may be quickly retrieved.

In addition, a user terminal group including two or more user terminals among the plurality of user terminals 100 may be formed. For example, if the user terminal A 101, the user terminal B 102, and the user terminal C 103 allow the video information and the product information to be shared with each other, the user terminal A 101 and the user terminal B 102. The user terminal C 103 may form a user terminal group for sharing product information. In this case, the user terminal 101 may search for the product image and the product information stored in the other user terminals 102 and 103 in the user terminal group.

The plurality of user terminals 100 may transmit product images and product information to the information management apparatus 300 through the communication network 200, respectively. The information management apparatus 300 includes an image search database DB, and classifies and stores product images and product information received from the plurality of user terminals 100. The information management apparatus 300 performs a task of matching and associating the merchandise image and the merchandise information stored in advance with the merchandise image and the merchandise information received from the plurality of user terminals 100. The information management apparatus 300 may construct a database with more rich and diverse data by receiving product images and product information from the plurality of user terminals 100.

When the information management apparatus 300 receives an image search request from one user terminal 101 among the plurality of user terminals 100, the information management device 300 receives information based on the product image or product keyword information received from the one user terminal 101. The product image and the product information stored in the management device 300 may be searched and the extracted product information may be provided to the user terminal 101. In this case, the one user terminal 101 may use the product image and the product information transmitted by the other user terminals 102, 103,... As the search database.

In addition, the information management apparatus 300 may extract shopping information related to each product and associate shopping information with product information. For example, the shopping information may include a sales store, a selling price, a lowest price, a highest price, an address of an online shopping mall site, and the like. Such shopping information may include various information necessary for the user to purchase the corresponding product. The information management apparatus 300 may obtain shopping information from various shopping information sites by accessing the internet network through the communication network 200.

2 is a conceptual diagram illustrating a configuration of a user terminal according to an embodiment of the present invention. Referring to FIG. 2, the user terminal includes a transceiver 201, an image photographing unit 202, an information manager 203, an input unit 204, and an information search unit 205.

The transceiver 201 is a communication interface that allows the user terminal 100 to communicate with the information management apparatus 300 through the communication network 200.

The image capturing unit 202 may include a digital camera lens or the like as a component for capturing an image of a product. An image of a product photographed using the image capturing unit 202 may be stored in the information manager 203.

The information manager 203 may generate an image thumbnail from the stored image of the product and extract feature information from the image thumbnail. For example, the information manager 203 may extract image feature information from a product image by using an edge algorithm and a grid algorithm. The former edge algorithm extracts the outer edge of the image (object), while the latter grid algorithm features feature points in the corner and convex regions of the image (object). ) Is extracted. The edge algorithm and the grid algorithm are well known in the field of imaging technology, and the specific extraction method is based on the known technology. The information manager 203 may classify and store the goods image and the goods information based on the feature information of the goods image.

The input unit 204 may be a component for inputting product information related to a product, and may include a keypad or a keyboard. The user may photograph an image of a product through the image capturing unit 202 and attach the product information related to the product through the input unit 204 by attaching the product image. For example, the product information may include user input information such as a product name, price information, a manufacturer, a release date, a purchase review, and the like.

The information retrieval unit 205 may retrieve the image and the product information of the product stored in the information management unit 203 in response to the user's request. The image and product information of the product may be stored in the form of a cache in the information management unit 203, so that the information retrieval unit 205 may quickly retrieve the image and product information of the product stored in the terminal by accessing the cache. have.

The user terminal 100 may request product information retrieval from the information management apparatus 300 through the transceiver 201 to retrieve product information using a database in the information management apparatus 300. The merchandise image photographed using the image capturing unit 202 may be transmitted to the information management apparatus 300 to search for merchandise information. In addition, to search for product information, additional information such as keyword information about a product, such as a product name, price information, a manufacturer, and a release date, is input through the input unit 204, and the information management device (201) through the transceiver unit 201 for product information search. 300). Thereafter, the user terminal 100 may receive the product information retrieved from the information management apparatus 300 through the transceiver 201.

3 is a conceptual diagram illustrating a configuration of an information management apparatus according to an embodiment of the present invention. The information management apparatus 300 includes a video information management unit 320 and a shopping information management unit 330 including a transceiver 310, a feature information extractor 321, a product information manager 322, and a product information searcher 323. ).

The information management apparatus 300 may communicate with the plurality of user terminals 100 through the communication network 200 using the transceiver 310. The information management apparatus 300 may receive product images and product information from the plurality of user terminals 100 through the transceiver 310.

The image information manager 320 performs a function of databaseting the image information and the related product information based on the product images and the product information received from the plurality of user terminals 100. The feature information extractor 321 generates an image thumbnail from the product image and extracts feature information from the image thumbnail. As described above, the feature information extraction may be performed in the same manner as the method performed by the user terminal 100. When the feature information is extracted from the user terminal 100, the feature information extraction apparatus 300 extracts the feature information. Sometimes work is unnecessary.

The product information manager 322 classifies and stores the feature information and the product information of the product image received from the plurality of user terminals 100 based on the feature information generated by the feature information extractor 321. The merchandise information management unit 322 performs merchandise information database by matching and associating the merchandise image information stored previously with the feature information and merchandise information of the merchandise image received from the plurality of user terminals 100. . As product images and product information are continuously received from the plurality of user terminals 100, the product information manager 322 may periodically update the product information database.

When a product search request is received from the user terminal through the transceiver 310, the product information search unit 323 performs the product information search and the feature information of the product image stored in the product information management unit 322. Since a product image and / or product information is received when a product search request is made by the user terminal, desired product information may be extracted by comparing and searching the received information with information stored in the product information management unit 322. Thereafter, the information management device 300 may transmit the retrieved product information to the user terminal 100 through the transceiver 310.

The shopping information manager 330 is connected to the image information manager 320 to search for and store shopping information related to a corresponding product. Here, the shopping information may include at least one of a sale shop, a sale price, a low price, a high price, and an online shopping mall site address. The shopping government manager 330 may access the internet network through the communication network 200 and extract shopping information related to the corresponding product. The shopping information extracted by the shopping information manager 330 may be transferred to the product information manager 320 and stored in association with the corresponding product information. Therefore, the information management apparatus 300 may transmit the shopping information related to the product together with the product information retrieved by the product information management unit 320 to the user terminal 100 through the transceiver 310.

4 is a flowchart illustrating an image search management method performed in an image search management system according to an embodiment of the present invention.

The user terminal 100 captures an image of the product (S101). The user terminal 100 stores and inputs product information related to the product together with the image of the product (S102). The user terminal 100 transmits the product image and the product information to the information management device 300 (S103).

The image information management unit 500 of the information management apparatus 300 receives the product image and the product information received from the user terminal 100, and classifies and stores the product information (S104). The product information database may be constructed by extracting feature information of product images received from the plurality of user terminals 100, determining whether the product is the same product, and classifying and storing the product information according to each product.

The user may start the product information search using the image of the product (S105). The user terminal 100 captures an image of a product to search for product information (S106). In this case, an image photographed in advance or an image obtained from another device may be used.

The user terminal 100 first searches a product image and product information stored in a cache form in its own terminal based on the product image at a high speed (S107). When searching for product information, a product-related keyword for searching for product information may be input in addition to the product image.

Next, the user terminal 100 may search for a product image and product information stored in another user terminal in a user terminal group including a plurality of preset user terminals. The user terminals in the user terminal group agree to share the product information in advance, and share the product image and the product information stored in the user terminal to be searchable to other user terminals in the user terminal group.

The user terminal 100 transmits a product image to the information management apparatus 300 after completing the search in the terminal and the search in the user terminal group and requests the product information search (S108). In this case, a product related keyword for product information search may be transmitted together with the product image transmission.

The image information manager 500 of the information management apparatus 300 extracts desired product information through feature information and product information search of a product image stored therein (S109). In addition, in order to generate shopping information related to the searched product, the shopping information manager 600 requests shopping information (S110).

The shopping information manager 600 extracts shopping information through shopping information search (S111), and transmits the extracted shopping information to the image information manager 500 (S112). The image information manager 500 transmits the product information and the related shopping information to the user terminal 100 together (S113).

5 is a flowchart illustrating an image search management method performed in a user terminal according to an embodiment of the present invention.

The user terminal 100 captures an image of the product (S501).

The user terminal 100 inputs and stores the product information related to the product together with the image of the product (S502).

The user terminal 100 generates an image thumbnail from the video image of the product, and extracts feature information from the image thumbnail (S503).

The product image and the product information are classified and stored based on the feature information of the extracted product image (S504). The merchandise image and the merchandise information classified and stored in this manner may be stored in a cache and used when searching for merchandise in the user terminal 100.

The user terminal 100 transmits the product image and the product information to the information management device 300 (S505). The product image and the product information transmitted to the information management apparatus 300 are databased in the information management apparatus.

The product information search procedure of the user terminal 100 may be performed in the same manner as described above with reference to FIG. 4.

6 is a flowchart illustrating an image search management method performed in an information management apparatus according to an embodiment of the present invention.

The information management apparatus 300 receives a product image photographed from the user terminal 100 and product information input by the user (S601).

The information management apparatus 300 generates an image thumbnail from the product image received from the user terminal 100, and extracts feature information from the image thumbnail (S602).

The information management apparatus 300 classifies and stores the received product image and the product information based on the extracted feature image of the product image. The information management apparatus 300 performs a product information database by classifying and storing product images and product information received from the plurality of user terminals 100.

Accordingly, the embodiments disclosed in the specification of the present invention are not intended to limit the present invention. The scope of the present invention should be construed according to the following claims, and all the techniques within the scope of equivalents should be construed as being included in the scope of the present invention.

The present invention can be widely used in the field using the image retrieval technology, graphics processing technology, object DB generation technology, wired and wireless network technology, etc., the image and information database (DB) registered by other users at the time of retrieval of the input image It can be used to diversify the search method and increase the search efficiency.

100: a plurality of user terminals
200: network
300: information management device
201: transceiver
202: the image capturing unit
203: Information Management Department
204: input unit
205: information retrieval unit
310: transceiver
320: video information management unit
321: feature information extracting unit
322: product information management unit
323: product information search unit
330: shopping information management unit

Claims (20)

A plurality of user terminals capable of capturing an image of a product, storing the image of the product and product information related to the product, transmitting the image of the product and the product information through a communication network, and searching the product information stored therein; And
The image and the product information of the product received from the plurality of user terminals are classified and stored, and the image and the product information of the product in response to the received image search request from one user terminal of the plurality of user terminals Information management device for searching and providing to the user terminal
Image search management system comprising a. Image capturing means for capturing an image of a product;
Input means for inputting product information related to the product;
Information management means for classifying and storing the image of the product and the product information; And
Transmitting the image of the product and the product information to a video information management device through a communication network, transmitting a request for product information search using the video information management device, and receiving the searched product information from the video information management device. Transceiver
User terminal comprising a. The user terminal according to claim 2, further comprising information retrieval means for retrieving product information stored in said information management means. The user terminal according to claim 3, wherein the image of the product and the product information are stored in a cache form in the information management means. The user terminal according to claim 2, wherein the information management means generates an image thumbnail from an image of the product, and extracts feature information from the image thumbnail. The user terminal of claim 2, wherein the product information includes at least one of a product name, price information, a manufacturer, a release date, and a purchase review. An information management apparatus connected with a plurality of user terminals through a communication network,
A transceiver for communicating with the plurality of user terminals;
A feature information extractor which generates an image thumbnail from images of goods received from the plurality of user terminals, extracts feature information from the image thumbnail, classifies the image of the goods and product information related to the goods An image information management device including a product information management unit to store the information; And
Shopping information management unit for searching for shopping information related to the product
Information management device comprising a. The apparatus of claim 7, wherein the image information management device further searches for and extracts product information to be provided to the user terminal based on an image of a product received from one user terminal among the plurality of user terminals. Information management apparatus comprising a. The information management apparatus of claim 7, wherein the shopping information comprises at least one of a sale shop, a sale price, a lowest price, a highest price, and an address of an online shopping mall site. The information management apparatus according to claim 7, wherein the product information management unit is updated periodically. In the video management method used in a system in which a plurality of user terminals and the information management device is connected through a communication network,
Photographing an image of the product and storing the image of the product and product information related to the product;
Transmitting an image of the product and the product information to the information management device; And
Generating, by the information management apparatus, classifying and storing the image of the product and the product information received from the plurality of user terminals and generating shopping information related to the product
Image management method comprising a. The method of claim 11,
Photographing an image of a search product by a user terminal of the plurality of user terminals;
Transmitting, by the user terminal, an image of the search product to the information management device; And
Extracting, by the information management device, the product information and the shopping information based on the received image of the search product from the user terminal and providing the same to the user terminal;
Image management method comprising a further. In the video management method used in a user terminal connected to the information management device through a communication network,
Photographing an image of a product and storing the image of the product and product information related to the product in the user terminal;
Generating an image thumbnail from the image of the product and extracting feature information from the image thumbnail;
Classifying and storing the image of the product and the product information by using the feature information in the user terminal; And
Transmitting the image of the product and the product information to the information management device;
Image management method comprising a. The method of claim 13, further comprising: searching for product information based on an image of the search product.
The product information retrieving step includes retrieving the image of the product and the product information stored in the user terminal. The method of claim 14, wherein the user terminal is connected to a plurality of other user terminals through the communication network,
The product information search step,
And after retrieving the image of the product and the product information stored in the user terminal, retrieving the image of the product and the product information stored in a predetermined user terminal group among the plurality of other user terminals. Video management method characterized in that. The method of claim 15, wherein the searching of the product information comprises:
And retrieving the product information after the retrieving the image and the product information of the product stored in the user terminal. The method of claim 16,
Receiving the product information from the information management device,
And the product information includes shopping information related to the product. In the video management method used in the information management device connected via a communication network with a plurality of user terminals,
Receiving an image of a product and product information related to the product from the plurality of user terminals;
Generating an image thumbnail from images of goods received from the plurality of user terminals and extracting feature information from the image thumbnail; And
Classifying and storing the image of the product and the product information by using the feature information
Image management method comprising a. The method of claim 18,
Receiving a product search request and an image of a search product from one user terminal among the plurality of user terminals;
Extracting the product information by using the image of the search product;
Extracting shopping information related to the search product; And
Transmitting the extracted product information and the shopping information to the one user terminal;
Image management method comprising a further. The method of claim 19, further comprising: receiving information related to a search product from one user terminal among the plurality of user terminals,
Extracting the product information and the shopping information is performed using the image of the search product and information related to the search product.

Images