JP4552100B2 - Product search system, product search device, and product search method for product search device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a product search system, a product search device, and a product search method for the product search device, and is suitable for application to window shopping using a network system, for example.
[0002]
[Prior art]
In recent years, digital cameras that record a captured image of a subject imaged via an optical system on a recording medium have become widespread, and the camera function is mounted on a mobile computer such as a portable notebook or PDA (Personal Digital Assistants). Things are also commercialized.
[0003]
If such a digital camera is carried, when a user performs shopping at a street corner, a desired product is imaged as necessary, and a captured image of the product is recorded on a recording medium in the digital camera. I can leave.
[0004]
Then, at a later date, the user determines whether or not to purchase the product based on the captured image of the desired product recorded on the recording medium and places an order with the seller.
[0005]
On the other hand, in recent years, the Web (that is, WWW: World Wide Web), which is an information network that enables various types of information in each server distributed on the Internet to be correlated and searched, has been widely used as an information service. By using the Web, the server on the merchant side exchanges data with the personal terminal on the merchandise purchaser side, so that the order reception process for the merchandise transmitted from the merchandise purchaser using the personal terminal is performed. It is thought that can be done.
[0006]
[Problems to be solved by the invention]
However, in such a product ordering service on the Web, when a user orders a desired product using his / her own personal terminal, it is necessary to input language related information on the desired product using a keyboard or the like.
[0007]
For this reason, the user must recognize in advance various product information such as the product name and manufacturer of the product shown in the captured image, and can obtain the desired product only from the captured image without any knowledge of product information. It was very difficult to order.
[0008]
In addition, digital camera functions are now being developed for mobile phones, PHS (Personal Handy Phone System), etc., and the frequency of use by users at street corners is becoming even higher. It is considered that the demand for product ordering services using captured images of products will increase.
[0009]
The present invention has been made in consideration of the above points, and an object of the present invention is to propose a product search system, a product search device, and a product search method for the product search device that make it possible to easily purchase a desired product.
[0010]
[Means for Solving the Problems]
In the present invention for solving the above problems, an image pickup image of the product, and a plurality of personal terminals transmitting via a predetermined network, and an information processing apparatus connected to the network, the information processing apparatus, a plurality Communication server that transmits / receives product image data to / from personal terminals over the network, an image search server that performs image search processing on product image data, and image interpretation processing based on the image search results searched by the image search server The database server that searches for a plurality of product candidates estimated according to the image interpretation processing, and the URL corresponding to the product image data that is stored according to the request of the personal terminal that stores the product image data. A server for resource distribution to send, and the image search server After dividing an image based on product image data into a plurality of regions, based on a plurality of extraction results obtained by extracting physical features of the center of gravity, area, shape and color distribution for each region, 1 The image search image generation means for generating an image search image composed of frame images for the number of frames, and the database server perform an image interpretation process based on the image search image data representing the image search image generated by the image search server A first delivery means for retrieving a plurality of product candidates to be processed, and sending product image data corresponding to the plurality of product images based on the result search candidates to the resource delivery server; A second sending means for sending a corresponding URL based on the data to the communication server together with the product image data; Nikeshon server was so based corresponding product image data provided and transmitting means for sending via the network the image data of the search result screen to the personal terminal.
[0011]
As a result, the user can easily purchase a desired product by transmitting an image of the product captured by himself / herself to a Web site established by the product distributor .
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0017]
(1) Configuration of Product Search System According to this Embodiment In FIG. 1, reference numeral 1 denotes a product search system according to this embodiment as a whole, and a plurality of servers 2A to 2D arranged by service providers providing product order services. And a personal information terminal 4 (4 ₁ to 4 _n ) used by a plurality of users are connected to each other via a network 5 and a site information management device 6 is connected to the server system 3. Is configured.
[0018]
The server system 3 is configured by interconnecting a plurality of servers including a communication server 2A, an image search server 2B, a database server 2C, and a resource distribution server 2D.
[0019]
As shown in FIG. 2, each of these servers 2A to 2D includes a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11 in which various programs are stored, and a RAM (Random Access Memory) as a work memory of the CPU 10. 12, a network interface 13 and a hard disk drive (HDD) 14 are connected to each other via an internal bus 15, and the network interface 13 is connected to the network 5 (FIG. 1) or another server (not shown). ).
[0020]
In this case, the CPU 10 takes in data and commands given from the personal terminal 4 accessed via the network 5 via the network interface 13, and performs various processes based on the data and commands and software stored in the ROM 11. Execute.
[0021]
Then, as a result of this processing, the CPU 10 sends, for example, screen data of a predetermined web page read from the hard disk device 14 and data such as other programs or commands to the corresponding personal terminal 4 via the network interface 13.
[0022]
In this manner, each of the servers 2A to 2D can transmit / receive screen data of the Web page and other necessary data to the personal terminal 4 that has accessed. Note that a plurality of databases (not shown) are stored in the hard disk devices 14 in the servers 2A to 2D so that necessary information can be read from the corresponding databases when various processes are executed. .
[0023]
In FIG. 1, each personal terminal 4 (4A _{1 to} 4A _n ) is a general personal computer equipped with a Web browser and a dedicated program for performing various processes related to a product ordering service as described below. Necessary data is transmitted and received by communicating with the personal terminal 4 and the servers 2A to 2D of the server system 2 via the network 5, and a web page screen based on the screen data obtained by the communication is displayed on the display. It is made to be able to.
[0024]
In FIG. 3, the communication server 2 </ b> A is a Web server that performs various processes when providing a product ordering service to a user, and transmits screen data of an image search Web page to the personal terminal 4 of the user. In addition, it can communicate with the image search server 2B and the database server 2C to exchange necessary data.
[0025]
The image search server 2B is a server equipped with an image search function for image data, and performs image search processing on the image data transferred from the communication server 2A so that the physical characteristics of the image are exposed. The result is sent to the database server 2C.
[0026]
The database server 2C performs image interpretation processing based on the image retrieval result given from the image retrieval server 2B, and retrieves a plurality of product candidates inferred from the image based on the original image data according to the interpretation processing result. To do.
[0027]
Subsequently, the database server 2C registers the URL (Uniform Resource Locator) of the Web site established by the seller as the selling source for each of the plurality of products as search candidates in the resource distribution server 2D in advance. After extracting from the URLs of a plurality of Web sites, the URLs of the extracted Web sites are sent to the personal terminal 4 via the communication server 2A.
[0028]
The database server 2C is a server on which a program for registering and managing the URL of each Web site is installed, and in response to various requests supplied from the site information management device 6, the resource server 2D uses resources. Image data related to the product (hereinafter referred to as product image data) is stored, or the URL of a Web site corresponding to each product image data is registered.
[0029]
The resource distribution server 2D stores product image data supplied from the site information management device via the database server, and responds to the determined product image data in response to a request supplied from each personal terminal. The screen data of the Web page screen corresponding to the URL of the seller is transmitted.
[0030]
(2) Product Search Processing Procedure In the product search system 1 actually shown in FIG. 3, each personal terminal 4 reads a captured image of a desired product previously captured by a user with a digital camera or the like from a recording medium. When the image data is displayed on the display and sent to the communication server 2A via the network 5, the product search processing procedure RT1 shown in FIG. 4 is started from step SP0.
[0031]
When the communication server 2A accesses an image search website via the network 5 based on the address designation by the personal terminal 4 possessed by the user, the communication server 2A displays a web page screen (hereinafter referred to as an image) corresponding to the entrance of the website. (Referred to as a search screen) (not shown) is displayed on the display of the personal terminal 4 (step SP1).
[0032]
The image search screen is provided with an image display field for displaying a captured image to be searched for prepared by the user, and a plurality of selection buttons are sequentially displayed in the vicinity of the image display field. The user can selectively execute a function assigned to the selection button by clicking the desired selection button with a mouse operation.
[0033]
The communication server 2A transmits the reception response of the image data D1 of the captured image transmitted from the personal terminal 4 possessed by the user via the network 5 to the personal terminal 4, and the image data D1 is directly transmitted to the image search server 2B. Send it out.
[0034]
The image search server 2B divides the image based on the image data D1 into a plurality of regions, and then extracts the barycentric position, area, shape, color distribution, and the like for each region, and based on the obtained plurality of extraction results. One frame image (hereinafter referred to as an image search image) is generated (hereinafter, this series of processing is referred to as image search processing). Thereafter, the image search server 2B sends image search image data D2 representing the image search image to the database server 2C (step SP2).
[0035]
The database server 2C is a server on which a program for registering and managing the URL of each Web site is installed, and in response to various requests supplied from the site information management device 6, the resource distribution server 2D uses resources. Image data (hereinafter referred to as “product image data”) D3 is stored, and URLs of websites corresponding to the product image data D3 are registered.
[0036]
The resource distribution server 2D accumulates product image data supplied from the site information management device 6 via the database server 2C, and determines the product image data determined in response to a request supplied from each personal terminal 4. Is sent to the database server 2C.
[0037]
The database server 2C performs image interpretation processing on the image retrieval image based on the image retrieval image data D2 given from the image retrieval server 2B, and is inferred from the image retrieval image according to the result of the interpretation processing. Search for multiple product candidates and list them in order from the most relevant product. Thereafter, the database server 2C sends product image data D3 corresponding to a plurality of products listed as search candidates to the resource delivery server 2D (step SP3).
[0038]
Based on the product image data D3 received from the database server 2C, the resource distribution server 2D has a plurality of URLs registered in advance for the URLs of websites established by sellers who sell the products. After extracting from the URL of the Web site, the URL of each extracted Web site is sent to the communication server 2A via the image search server 2B together with the corresponding product image data D3 (step SP4).
[0039]
The communication server 2A is a Web page screen (hereinafter referred to as a search result screen) for constructing a so-called hyper link based on URLs of a plurality of Web sites obtained from the database server 2C and corresponding product image data. After calling (not shown), the screen data D4 of the search result screen is sent to the corresponding personal terminal 4 via the network (step SP5).
[0040]
Thus, the corresponding personal terminal 4 displays a search result screen based on the screen data D4 transmitted from the communication server 2A via the network on the display, whereby a plurality of URLs listed on the search result screen and By selectively clicking the URL corresponding to the desired product image from the product images corresponding to these with the mouse, it is possible to access the Web site associated with the URL.
[0041]
(3) Operation and effect of the present embodiment In the above-described configuration, in the product search system 1, the user accesses the server system 3 owned by the service provider via the network 5 using the personal terminal 4, and the server A captured image of a desired product is transmitted to the server system 3 using an image search screen that is a Web page screen obtained from the system 3.
[0042]
In this server system 3, an image search process is performed on the captured image obtained from the personal terminal 4, and physical features such as the position of the center of gravity, area, shape, and color distribution are obtained for each area obtained by dividing the captured image. An image search image including a frame image obtained by extracting and synthesizing the plurality of extraction results is generated.
[0043]
In this way, the server system 3 performs image interpretation processing on the image search image, searches a plurality of product candidates inferred from the image search image, and enumerates the products in order from the most relevant product, For each product, the URL of a Web site established by the sales agent serving as the selling source is extracted and transmitted to the personal terminal 4 together with the image data of the corresponding product.
[0044]
As a result, in the corresponding personal terminal 4, while viewing the image search screen displayed on the display, a search result screen created based on URLs of a plurality of Web sites and product image data corresponding to these URLs Is displayed.
[0045]
Thus, in the personal terminal 4, the URL corresponding to the desired product image is selectively clicked with the mouse from the plurality of URLs listed on the search result screen displayed on the display and the corresponding product images. Thus, the website associated with the URL can be accessed.
[0046]
Therefore, in this personal terminal 4, a Web page screen obtained via the network from the accessed Web site is displayed on the display, and as a result, the user opens the merchant of the product from only the image of the product captured by himself / herself. By accessing the website, the product can be ordered and purchased.
[0047]
According to the above configuration, in this product search system, when a user transmits a captured image of a desired product to a server system on a Web page screen provided in response to an access request from a service provider using the personal terminal 4. The server system 3 performs image search processing and image interpretation processing on the captured image to search a plurality of product candidates inferred from the captured image, and lists them in order from the most relevant product. By extracting the URL of the website opened by the seller as the seller and sending it to the corresponding personal terminal 4 together with the image data of the corresponding product, a plurality of URLs are displayed on the display of the personal terminal 4. URLs and product images corresponding to these URLs can be displayed as a Web page screen. As a result, the user can display the Web page screen. By specifying a URL corresponding to a desired product image by using it, it is possible to access a Web site associated with the URL, thus realizing a product search system 1 that makes it possible to easily purchase a desired product. it can.
[0048]
(4) Other Embodiments In the above-described embodiment, the case where a general personal computer is applied as the personal terminal (terminal) 4 (4 ₁ to 4 _n ) has been described. The present invention is not limited to this, and can be widely applied to various terminals such as a mobile phone and a mobile terminal as long as they can be accessed via a network.
[0049]
In the above-described embodiment, a server system (communication server 2A, image search server 2B, database server 2C, and resource distribution server) arranged by a personal terminal (terminal) 4 (4 ₁ to 4 _n ) and a service provider. 2D) Although the case where the information processing device and the product search device 3 are connected to each other via the network 5 such as the Internet has been described, the present invention is not limited to this, and other general methods are also used. The present invention can be widely applied not only to a wired communication network such as a public line or a LAN (Local Area Network) but also to a network including a wireless communication network.
[0050]
Further, in the above-described embodiment, as an image to be transmitted from the terminal to the server system (information processing apparatus, product search apparatus) 3, the user captures in advance with a digital camera or the like, and the personal terminal (terminal) 4 (4 ₁ to 4 _n) has been described with the case of applying a desired captured image of the product that has been recorded on a recording medium which is provided or externally loaded therein, the present invention is not limited thereto, the desired product If it is an image represented, an image obtained by reading a poster representing a product or a picture very close to a real photograph with a scanner may be used.
[0051]
Furthermore, in the above-described embodiment, the case where the storage unit 2D of the server system 3 is applied as the storage unit that stores the product images representing various products has been described, but the present invention is not limited thereto. In short, as long as the product image to be searched can be uniquely read out, various other configurations may be applied as the storage means.
[0052]
Further, in the above-described embodiment, based on an image received via the network 5, at least one or more product images that match or are similar to the image are selected from among product images representing various products stored in advance. Although the case where the image search server 2B of the server system 3 is applied as the search means for searching has been described, the present invention is not limited to this, and the point is that a product image that matches or is similar based on the received image. As long as at least one search can be performed, various other configurations can be widely used as the search means.
[0053]
In this case, as a search method, the image search process as described above, that is, the image is divided into each region, and physical features such as the centroid position, area, shape, and color distribution are extracted for each region, Although the case where the method of generating an image search image composed of frame images obtained by combining the plurality of extraction results has been described, the present invention is not limited to this, and in short, the physical feature of the received image is extracted. In this way, if it can be used as a search target, it can be widely applied to various other search methods.
[0054]
Further, in the above-described embodiment, for each product image searched as matching or similar to the received image, the address on the network for the site established by the product seller corresponding to each product image is set. The case where the transmission means for transmitting to the terminal via the network is configured from the communication server 2A and the database server 2C of the server system 3 has been described. However, the present invention is not limited to this, and in short, the terminal has the terminal. If the user can transmit information that allows access to a site established by the seller of the product related to the image transmitted by the user, various other configurations may be applied as the transmission means. .
[0055]
Further, in the above-described embodiment, the case where the product search system 1 according to the present invention is applied to image purchase based on an image has been described. However, the present invention is not limited to this, and a landscape photographed at a street corner or a travel destination. The present invention can also be applied to a system that searches various information such as a place or a store related to the scenery based on the image.
[0056]
In addition, if the person is a celebrity based on the face image of the person taken at the street corner, various information such as the activity content related to the person can be searched, while the criminal who the person is requesting If so, the criminal search site established by the police can be accessed to contribute to the search target. Furthermore, even if a general person has a homepage, the person's homepage can be accessed based on the captured face image.
[0057]
【The invention's effect】
As described above, according to the present invention, in the product search system, the information processing apparatus divides an image based on product image data into a plurality of regions based on an image transmitted from a user's terminal, and then, for each region. Based on a plurality of extraction results obtained by extracting physical features of the center of gravity position, area, shape, and color distribution, an image search image composed of one frame image is generated, and a plurality of image search images are generated from the image search image. The product image is estimated and the product candidate is transmitted to the user terminal together with the URL, so that the user can specify the product from the received plurality of product images and purchase the desired product. A product search system that makes it possible to easily purchase products.
[0058]
Further, according to the present invention, in the product search device, based on the image transmitted from the user's terminal , after dividing the image based on the product image data into a plurality of regions, the center of gravity position, area, shape and Based on a plurality of extraction results obtained by extracting physical characteristics of the color distribution, an image search image composed of one frame image is generated, and a plurality of product images are estimated from the image search image Since the product candidate is transmitted to the user's terminal together with the URL, the user can purchase the desired product by specifying the product from the received plurality of product images, so that the desired product can be easily purchased. A product search device can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a product search system to which the present invention is applied.
2 is a block diagram showing a schematic configuration of each server shown in FIG. 1. FIG.
3 is a block diagram showing a detailed configuration of the product search system shown in FIG.
FIG. 4 is a flowchart showing a product search processing procedure.
[Explanation of symbols]
1 ...... product search system, 2A ...... communication server, 2B ...... images server, 2C ...... database server, 2D ...... resource delivery server, 3 ...... server _{system, 4 (4 1 ~4 n)} ...... Personal terminal, 5... Network, 6... Site information management device, RT 1.

Claims (5)

A plurality of personal terminals that transmit a captured image of a product via a predetermined network;
An information processing device connected to the network,
The information processing apparatus
A communication server that transmits / receives product image data to / from the plurality of personal terminals via the network;
An image search server for performing an image search process on the product image data;
A database server that searches a plurality of product candidates estimated according to the image interpretation process by performing an image interpretation process based on the image search result searched by the image search server;
A resource delivery server for storing the product image data and transmitting a URL corresponding to the product image data determined in response to a request from the personal terminal to the database server;
The image search server divides an image based on the image data of the product into a plurality of regions, and then extracts a plurality of physical features obtained by extracting physical features of the centroid position, area, shape, and color distribution for each region. An image search image generation means for generating an image search image consisting of one frame image based on the extraction result;
The database server performs image interpretation processing based on the image search image data representing the image search image generated by the image search server and searches for a plurality of candidate products to be estimated, and based on the search candidates as a result First sending means for sending the product image data corresponding to a plurality of product images to a resource distribution server;
The resource delivery server, a second sending means for sending the URL to the communication server together with the product image data based on the product image data;
The communication server is the corresponding goods retrieval system Ru comprising a transmitting means image data of the search result screen on the basis of product image data transmitted via the network to the personal terminal. The transmission means is
Together with the address on said network for website vendor of goods in accordance with the above product image data is established, respectively, each product image data, and transmits against the said personal terminal via the network
Product search system according to 請 Motomeko 1. A communication server that transmits and receives image data to and from the plurality of personal terminals via a network;
An image search server for performing an image search process on the image data of the product;
A database server that performs an image interpretation process based on the image search result searched by the image search server, and searches for a plurality of product candidates estimated according to the image interpretation process;
A resource delivery server that stores product image data and transmits a URL corresponding to the product image data determined in response to a request from the personal terminal to the database server;
The image search server divides an image based on the image data of the product into a plurality of regions, and then extracts a plurality of physical features obtained by extracting physical features of the centroid position, area, shape, and color distribution for each region. An image search image generation means for generating an image search image consisting of one frame image based on the extraction result;
The database server performs image interpretation processing based on the image search image data representing the image search image generated by the image search server and searches for a plurality of candidate products to be estimated, and based on the search candidates as a result First sending means for sending the product image data corresponding to a plurality of product images to a resource distribution server;
The resource delivery server, a second sending means for sending a URL corresponding to the product image data together with the product image data to the communication server;
The communication server is the corresponding product image data based on the search result screen goods retrieval device Ru comprising a transmitting means for transmitting via the network to the personal terminal the image data of. The transmission means is
Together with the address on said network for website vendor of goods in accordance with the above product image data is established, respectively, each said product image data, and transmits against the said personal terminal via the network
Product search device according to 請 Motomeko 3. A first step of transmitting / receiving image data to / from the plurality of personal terminals via a network by a communication server;
A second step of performing an image search process on the image data of the product by an image search server;
A third step of performing, by the database server, image interpretation processing based on the image search result searched by the image search server, and searching for a plurality of product candidates estimated according to the image interpretation processing;
A fourth step of storing product image data by a resource distribution server and transmitting a URL corresponding to the product image data determined in response to a request from the personal terminal to the database server;
The second step includes dividing the image based on the image data of the product into a plurality of regions, and then extracting the physical features of the centroid position, area, shape, and color distribution for each region. Based on the extraction result, generate an image search image consisting of one frame image,
In the third step, based on the image search image data representing the image search image generated by the image search server, image interpretation processing is performed to search a plurality of product candidates to be estimated, and the search candidates are obtained as results. Send the product image data according to the plurality of product images based on the resource distribution server,
The fourth step sends the corresponding URL based on the product image data to the communication server together with the product image data,
The first step transmits image data of a search result screen to the personal terminal based on the corresponding product image data via the network.
Product search method for product search device .

Images