KR101787400B1 - Method for providing image management based on user information and device and system using the same - Google Patents

Abstract

The present invention provides a method, an apparatus, and a related system capable of increasing a search efficiency by forming a database of a video object database by user participation and diversifying the information by user input and diversifying a search method.
According to the present invention, image information and product information are converted into a database by transmitting a product image and related product information shot by a plurality of users to an information management apparatus, and information of a plurality of users can be utilized as a search database .

Description

Technical Field [0001] The present invention relates to a user information based image search management method and a device and system using the same,

The present invention relates to an image search management method for searching an input image and managing a database when searching for a mobile phone or a web, and an apparatus using the same. More particularly, the present invention relates to a method for diversifying a search method and increasing search efficiency by utilizing an image and information database (DB) registered by other users at the time of searching an input image, Search technology, graphics processing technology, object database creation technology, wired / wireless network technology, and the like.

Conventional image recognition and retrieval techniques involve a process of detecting feature information from a video image and comparing the detected value with a stored database to retrieve the information. Recently, the development of camera technology and image processing technology and the spread of smart phones and web cams equipped with high-performance cameras are increasingly promoting research on image retrieval methods.

In the conventional image retrieval method, an image is captured using a camera such as a PC camera or a mobile phone, and a feature point is extracted from an edge or a grid in the acquired image and compared with a registered feature information database To retrieve image recognition and related information.

However, in the conventional image retrieval technology, even if the images of the same object are extracted due to the change of the illumination, the change of the angle, the rotation of the object, the difference of the object background, etc., There are many. In addition, there is a problem that the image retrieval result is different for the same object due to the difference of the photographing tool and the photographer. In addition, there is a great deal of work to continually update newly created images and to make them into a database, so that an integrated solution for searching all images is not completed.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a method and apparatus for increasing the retrieval efficiency by forming a database of image objects by user participation, And a related system.

According to a first aspect of the present invention, an image search management system according to the present invention is an image search management system for shooting an image of a product, storing image of the product and product information related to the product, A plurality of user terminals capable of transmitting the goods information through a communication network and searching for the goods information stored therein; And an image processing unit for classifying and storing the image of the goods and the goods information received from the plurality of user terminals and storing the image of the goods and the goods information in response to a received image search request from one user terminal among the plurality of user terminals And provides the information to the one user terminal.

According to a second aspect of the present invention, there is provided a user terminal comprising: image capturing means for capturing an image of a commodity; Input means for inputting product information related to the product; Information management means for classifying and storing the image of the goods and the goods information; And transmitting the image of the goods and the goods information to the image information management apparatus through a communication network, transmitting a request for product information search using the image information management apparatus, receiving the goods information retrieved from the image information management apparatus, And a transmitting / receiving unit for transmitting / receiving data.

And information retrieval means for retrieving the goods information stored in the information management means.

And the image of the goods and the goods information are stored in the form of a cache in the information management means.

The information management means generates an image thumbnail from the image of the goods and extracts the feature information from the image thumbnail.

The product information includes at least one of a product name, price information, a manufacturer, a release date, and a purchase end date.

According to a third aspect of the present invention, an information management apparatus connected through a communication network with a plurality of user terminals according to the present invention comprises: a transmission / reception unit for communicating with the plurality of user terminals; A feature information extracting unit that generates an image thumbnail from the image of the product received from the plurality of user terminals and extracts the feature information from the image thumbnail, and classifies the product image related to the product and the product A product information management unit for storing the product information; And a shopping information management unit for searching shopping information related to the goods.

The image information managing apparatus further comprises a goods information searching unit for searching and extracting the goods information to be provided to the one user terminal based on the image of the received goods from one user terminal among the plurality of user terminals .

The shopping information includes at least one of a sales store, a selling price, a lowest price, a highest price, and an address of an online shopping mall site.

And the product information management unit is periodically updated.

According to a fourth aspect of the present invention, there is provided an image management method used in a system in which a plurality of user terminals and an information management apparatus are connected through a communication network according to the present invention, Storing product information; Transmitting the image of the goods and the goods information to the information management apparatus; And the information management apparatus classifies and stores the image of the goods and the goods information received from the plurality of user terminals and generates shopping information related to the goods.

Capturing an image of a search product by a user terminal of the plurality of user terminals; The one user terminal transmitting the image of the search product to the information management apparatus; And extracting the goods information and the shopping information based on the image of the search goods received from the one user terminal, and providing the extracted goods information and the shopping information to the user terminal.

According to a fifth aspect of the present invention, there is provided an image management method used in a user terminal connected to an information management apparatus through a communication network according to the present invention, comprising the steps of capturing an image of a product, Storing in a user terminal; Generating an image thumbnail from the image of the product and extracting feature information from the image thumbnail; Classifying the image of the goods and the goods information using the feature information and storing the classified image in the user terminal; And transmitting the image of the goods and the goods information to the information management apparatus.

And searching for the product information based on the image of the search product, wherein the product information searching step includes searching for the image of the product and the product information stored in the user terminal.

Wherein the user terminal is connected to a plurality of other user terminals via the communication network and the product information searching step includes a step of searching for the image of the goods and the goods information stored in the user terminal, And retrieving the image of the goods and the goods information stored in the predetermined user terminal group among the terminals.

The merchandise information searching step may further include a step of requesting the information managing apparatus to search for merchandise information after searching for the merchandise image and the merchandise information stored in the user terminal.

Further comprising the step of receiving the goods information from the information management apparatus, wherein the goods information includes shopping information related to the goods.

According to a sixth aspect of the present invention, there is provided a video management method used in an information management apparatus connected to a plurality of user terminals via a communication network, the video management method comprising: Receiving; Generating an image thumbnail from the images of the goods received from the plurality of user terminals and extracting feature information from the image thumbnails; And classifying and storing the image of the goods and the goods information using the feature information.

Receiving an image of a product search request and a search product from a user terminal of the plurality of user terminals; Extracting the product information using the image of the search product; Extracting shopping information related to the search product; And transmitting the extracted product information and the shopping information to the one user terminal.

The method of claim 1, further comprising receiving information related to a search product from a user terminal of the plurality of user terminals, wherein the product information and the shopping information are extracted using information related to the search product and the image of the search product .

According to the present invention, an active search method based on user participation rather than a conventional passive search method is provided, and a video captured by a user is converted into an image object database and registered in a server, .

In addition, according to the present invention, various images and information due to user participation are converted into a database and the search method is diversified, thereby increasing search efficiency.

In addition, according to the present invention, it is possible to save the cost of constructing a video object database by sharing data by user participation, and to provide a convenient and various search method and a user-centered interface for a user in product search .

1 is a conceptual diagram illustrating a configuration of an image search management system according to an embodiment of the present invention;
2 is a conceptual diagram illustrating a configuration of a user terminal according to an embodiment of the present invention;
3 is a conceptual diagram illustrating a configuration of an information management apparatus according to an embodiment of the present invention;
4 is a flowchart illustrating an image search management method performed in an image search management system according to an embodiment of the present invention.
5 is a flowchart illustrating an image search management method performed by a user terminal according to an exemplary embodiment of the present invention.
6 is a flowchart illustrating an image search management method performed by an information management apparatus according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and the operation and effect thereof will be clearly understood through the following detailed description.

Before describing the present invention in detail, the same components are denoted by the same reference symbols as possible even if they are displayed on different drawings. In the case where it is judged that the gist of the present invention may be blurred to a known configuration, do.

1 is a conceptual diagram illustrating a configuration of an image search management system according to an embodiment of the present invention. Referring to FIG. 1, an image search management system includes a plurality of user terminals 100, a communication network 200, and an information management apparatus 300.

The plurality of user terminals 100 may include various types of user terminals capable of transmitting and receiving data through the communication network 200 and taking images. For example, the plurality of user terminals 100 include various mobile communication terminals including a smart phone, a PC, a PDA, a digital camera, a web cam, a fixed or mobile subscriber unit, or any other type of device capable of operating in a wired and wireless environment . The plurality of user terminals 100 can communicate with each other through the communication network 200 and the plurality of user terminals 100 can communicate with the information management apparatus 300 through the communication network 200.

The communication network 200 includes a plurality of user terminals 100 and various networks for providing wired / wireless communication to the information management apparatus 300. For example, the communication network 200 may be a wired Internet network, and may include a wireless data network connected through a mobile communication network (CDMA, WCDMA, WiBro, etc.), or an Internet network connected through a short distance communication. Specifically, the CDMA network may be connected to the data network through a PDSN (Packet Data Serving Node), and the W-CDMA network may be connected to the data network through a GGSN (Gateway GPRS Support Node). In addition, in an area such as a hot spot that can be connected to an access point (AP), it can be connected to the Internet network through short-distance communication such as Bluetooth or Wi-Fi.

Such an Internet network is a network existing in the TCP / IP protocol and its upper layer. The Internet network includes a network such as Hypertext Transfer Protocol (HTTP), Telnet, File Transfer Protocol (FTP), Domain Name Server (DNS), Simple Mail Transfer Protocol Simple Network Management Protocol (SNMP), Network File Service (NFS), and Network Information Service (NIS).

Each of the plurality of user terminals 100 can take a picture of a commodity using a camera, a web cam, or the like mounted on the user terminal. In addition to storing photographed images, it is possible to input product information related to the product together with the image of the product. For example, a user may further store user input information about a product together with an image of the product using the user terminal 100. [ The user input information may include, for example, a product name, price information, a manufacturer, a release date, a purchase date, and the like.

Meanwhile, the product image mentioned below may be a photographic image of a product directly photographed using the user terminal 100, but it is not limited thereto, and it may be a video image obtained through a web site or an image acquired through another terminal .

The plurality of user terminals 100 may store product images and product information in the terminal, respectively. For example, the plurality of user terminals 100 store the product image and the product information in a cache form so that the product image and the product information stored in the terminal can be searched at a high speed when performing the image search.

Also, a user terminal group including two or more user terminals among a plurality of user terminals 100 may be formed. For example, if the user terminal A 101, the user terminal B 102, and the user terminal C 103 are allowed to share image information and product information, the user terminal A 101, the user terminal B 102, , And the user terminal C (103) can form a user terminal group for product information sharing. At this time, the user terminal 101 can search for goods images and product information stored in other user terminals 102 and 103 in the user terminal group.

The plurality of user terminals 100 may transmit product images and product information to the information management apparatus 300 through the communication network 200, respectively. The information management apparatus 300 includes an image search database (DB), and classifies and stores product images and product information received from a plurality of user terminals (100). The information management apparatus 300 performs an operation of matching and associating the product image and the product information stored in advance with the product image and the product information received from the plurality of user terminals 100. [ The information management apparatus 300 can receive a commodity image and commodity information from a plurality of user terminals 100 to build a database with richer and diverse data.

The information management apparatus 300 receives the image search request from one user terminal 101 of the plurality of user terminals 100 and displays the information on the basis of the product image or the product keyword information received from one user terminal 101 It is possible to search for the goods image and the goods information stored in the management apparatus 300 and to provide the extracted goods information to the one user terminal 101. [ At this time, the one user terminal 101 can use the product image and the product information transmitted from the other user terminals 102, 103, ... to the information managing apparatus 300 as a search database.

Also, the information management apparatus 300 can extract shopping information related to each product and associate shopping information with product information. For example, the shopping information may include a sales store, a selling price, a lowest price, a highest price, an address of an online shopping mall site, and the like. Such shopping information may include various information necessary for the user to purchase the product. The information management apparatus 300 can access the Internet network through the communication network 200 and obtain shopping information from various shopping information sites.

2 is a conceptual diagram illustrating a configuration of a user terminal according to an embodiment of the present invention. 2, the user terminal includes a transceiver 201, an image capturing unit 202, an information managing unit 203, an input unit 204, and an information searching unit 205.

The transmission and reception unit 201 is a communication interface that allows the user terminal 100 to communicate with the information management apparatus 300 through the communication network 200. [

The image photographing unit 202 may include a digital camera lens or the like as a component for photographing a product image. The image of the product photographed using the image photographing unit 202 may be stored in the information managing unit 203. [

The information management unit 203 may generate an image thumbnail from the image of the stored product and extract feature information from the image thumbnail. For example, the information management unit 203 may extract image feature information from a product image using an edge algorithm and a grid algorithm. The former edge algorithm extracts the outer edge of the image and the latter grid algorithm uses the feature point in the corner of the image and the convex region. ) Is extracted. The edge algorithm and the grid algorithm are well known in the field of image technology, and the specific extraction method is based on known techniques. The information management unit 203 can classify and store the product image and the product information based on the feature information of the product image.

The input unit 204 may include a keypad, a keyboard, or the like as a component for inputting product information related to a product. The user can input product information related to the product through the input unit 204 after attaching the product image to the product image after photographing the product through the image photographing unit 202. [ For example, the product information may include user input information such as a product name, price information, a manufacturer, a release date, a purchase date, and the like.

The information retrieving unit 205 can retrieve the image and the product information of the goods stored in the information managing unit 203 in response to the user's request. The video and product information of the goods can be stored in a cache form in the information management unit 203. The information searching unit 205 can access the cache and quickly search for the image and the goods information of the goods stored in the terminal have.

The user terminal 100 may request the information management apparatus 300 to search for product information through the transmission / reception unit 201 in order to retrieve the product information using the database in the information management apparatus 300. [ It is possible to transmit the product image photographed using the image photographing unit 202 to the information management apparatus 300 for retrieving the product information. In addition, in order to search for product information, additional information such as keyword information about a product such as a product name, price information, manufacturer, and release date is input through the input unit 204 and the information management apparatus 300). Thereafter, the user terminal 100 can receive the goods information retrieved from the information management apparatus 300 through the transmission / reception unit 201. [

3 is a conceptual diagram illustrating a configuration of an information management apparatus according to an embodiment of the present invention. The information management apparatus 300 includes a transmission information receiving unit 310, a characteristic information extracting unit 321, a video information managing unit 320 including a product information managing unit 322 and a product information searching unit 323, ).

The information management apparatus 300 can communicate with a plurality of user terminals 100 through the communication network 200 using the transmission / reception unit 310. The information management apparatus 300 can receive product images and product information from a plurality of user terminals 100 through the transmission / reception unit 310.

The image information management unit 320 performs a function of converting the image information and related product information into a database based on the product image and the product information received from the plurality of user terminals 100. [ The feature information extracting unit 321 generates an image thumbnail from the product image and extracts the feature information from the image thumbnail. The extraction of the feature information may be performed in the same manner as the method performed in the user terminal 100 as described above. When the feature information is extracted from the user terminal 100, the information management apparatus 300 extracts feature information Sometimes the work is unnecessary.

The product information management unit 322 classifies and stores the feature information and the product information of the product image received from the plurality of user terminals 100 based on the feature information generated by the feature information extraction unit 321. [ The product information management unit 322 performs the product information database conversion by matching the existing product image information with the feature information and the product information of the product image received from the plurality of user terminals 100 . The product information management unit 322 can periodically update the product information database as the product image and the product information are continuously received from the plurality of user terminals 100. [

When the product search request is received from the user terminal through the transmission / reception unit 310, the product information search unit 323 searches for the feature information and the product information of the product image stored in the product information management unit 322. Since the product image and / or the product information are received upon the product search request of the user terminal, the desired product information can be extracted by comparing the received information and the information stored in the database in the product information management unit 322. Thereafter, the information management apparatus 300 can transmit the searched product information to the user terminal 100 through the transmission / reception unit 310. [

The shopping information management unit 330 is connected to the image information management unit 320 to retrieve and store shopping information related to the goods. Here, the shopping information may include at least one of a sales store, a selling price, a lowest price, a highest price, and an online shopping mall site address. The shopping management unit 330 can access the Internet network through the communication network 200 and extract shopping information related to the goods. The shopping information extracted by the shopping information management unit 330 may be transmitted to the product information management unit 320 and stored in association with the corresponding product information. Accordingly, the information management apparatus 300 can transmit the shopping information related to the product together with the product information retrieved from the product information management unit 320 to the user terminal 100 through the transmission / reception unit 310. [

4 is a flowchart illustrating an image search management method performed in the image search management system according to an embodiment of the present invention.

The user terminal 100 captures an image of a commodity (S101). The user terminal 100 inputs and stores product information related to the product together with the image of the product (S102). The user terminal 100 transmits the product image and the product information to the information management apparatus 300 (S103).

The image information management unit 500 of the information management apparatus 300 receives the product image and the product information received from the user terminal 100, and classifies and stores the product information (S104). It is possible to extract the feature information of the product images received from the plurality of user terminals 100 to determine whether they are the same product or not and classify and store the product information according to each product.

The user can start retrieving the product information using the image of the product (S105). The user terminal 100 captures an image of a commodity for retrieving the commodity information (S106). At this time, an image taken in advance or an image obtained from another apparatus can be used.

The user terminal 100 searches the product image and the product information stored in the cache form in the terminal at a high speed based on the product image (S107). A product-related keyword for searching for the product information can be input by attaching to the product image at the time of retrieving the product information.

Next, the user terminal 100 can search for goods images and goods information stored in other user terminals in a user terminal group including a plurality of user terminals set in advance. The user terminals in the user terminal group consent to sharing the product information in advance and share the product image and the product information stored in the terminal in a manner that they can be searched to other user terminals in the user terminal group.

After completing the search within the terminal and the search within the user terminal group, the user terminal 100 transmits the product image to the information management apparatus 300 and requests the product information search (S108). At this time, it is possible to transmit a product related keyword for product information search together with the product image transmission.

The image information management unit 500 of the information management apparatus 300 extracts desired product information through the feature information and the product information search of the product image stored therein (S109). In addition, the shopping information management unit 600 requests the shopping information to generate shopping information related to the searched product (S110).

The shopping information management unit 600 extracts the shopping information through the shopping information search (S111) and transmits the extracted shopping information to the image information management unit 500 (S112). The video information management unit 500 transmits the product information and the related shopping information together to the user terminal 100 (S113).

5 is a flowchart illustrating an image search management method performed by a user terminal according to an exemplary embodiment of the present invention.

The user terminal 100 captures an image of a commodity (S501).

The user terminal 100 inputs and stores product information related to the product together with the image of the product (S502).

The user terminal 100 generates an image thumbnail from the image of the goods and extracts the feature information from the image thumbnail (S503).

The product image and the product information are classified and stored based on the extracted feature information of the product image (S504). The product image and product information classified and stored in this manner can be stored in a cache form and used in product search in the user terminal 100.

The user terminal 100 transmits the product image and the product information to the information management apparatus 300 (S505). The product image and the product information transmitted to the information management apparatus 300 are converted into a database in the information management apparatus.

The product information retrieval procedure of the user terminal 100 may be performed in the same manner as described above with reference to FIG.

6 is a flowchart illustrating an image search management method performed by the information management apparatus according to an exemplary embodiment of the present invention.

The information management apparatus 300 receives the product image photographed from the user terminal 100 and the product information input by the user (S601).

The information management apparatus 300 generates an image thumbnail from the product image received from the user terminal 100, and extracts the feature information from the image thumbnail (S602).

The information management apparatus 300 classifies and stores the received product image and product information based on the extracted feature information of the product image. The information management apparatus 300 classifies and stores merchandise image and merchandise information received from a plurality of user terminals 100, thereby performing merchandise information database conversion.

Accordingly, the embodiments disclosed in the specification of the present invention are not intended to limit the present invention. The scope of the present invention should be construed according to the following claims, and all the techniques within the scope of equivalents should be construed as being included in the scope of the present invention.

The present invention can be widely used in fields using an image retrieval technology, a graphic processing technology, an object DB creation technology, a wire / wireless network technology, and the like. The search method can be diversified and the search efficiency can be increased.

100: a plurality of user terminals
200: Network
300: Information management device
201: Transmitting /
202:
203:
204:
205: Information retrieval unit
310: Transmitting /
320:
321: Feature information extracting unit
322:
323: Product information search section
330: shopping information management unit

Claims (1)

In a user terminal,
An image capturing unit for capturing an image of a product;
An input unit for inputting product information related to the product;
An information management unit for classifying and storing the image of the goods and the goods information;
And transmitting the image of the goods and the product information to the image information management apparatus through a communication network, transmitting a request for product information search using the image information management apparatus, and receiving product information retrieved from the image information management apparatus A transmission / reception unit; And
And an information retrieval unit for retrieving and providing the image of the goods and the goods information classified and stored by the information management unit when the user requests the user terminal group including the user terminal.

Images