JP4058288B2 - Similar image presentation system and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a similar image presentation system and method for searching for an image similar to an image designated by a user terminal that has been accessed via a network and for browsing on the user terminal.
[0002]
[Prior art]
In an auction or shopping on the Internet, a product image is generally displayed to a user together with a description of product content. In this case, as one of the needs of the user, there is a function of presenting a product having an appearance similar to that when one product is selected. For example, in the case of merchandise such as clothes and ornaments, the appearance of colors, shapes, patterns, etc. can be a great motivation for the merchandise. Therefore, if other products similar to the appearance of the product being browsed by the user can be displayed on the auction site or shopping site, it is effective in increasing the user's willingness to purchase.
[0003]
[Problems to be solved by the invention]
However, the effects as described above by presenting similar images have not been recognized so far, and a service for displaying other products similar in appearance to a certain product on a site on the Internet has not been provided.
The present invention has been made in view of the above points, and an object thereof is to present an image similar to an image designated by a user to a user terminal that has been accessed via a network.
[0004]
[Means for Solving the Problems]
In order to achieve the above object, the invention described in claim 1 is configured to search for an image similar to a product image designated by a user terminal that has configured a product sales site and accessed via a network. A similar image presentation system for browsing on the user terminal, in which an image database storing image data of various products sold on the product sales site and product information of the various products is stored for each category. For each image data stored in the product database and each image data stored in the image database, an image ID is associated with feature data representing the feature of the image, and according to the category of the product represented by the image in the product database. a feature database recorded by division, category in the product database of the corresponding products with the image ID Specified by the received search request from the feature database, a category request database for storing category data indicating the correspondence relationship between the user database, a search request receiving means for receiving a search request for a similar image with an image ID specified from the user terminal. The feature data acquisition means for acquiring the feature data associated with the image ID, the category corresponding to the image ID specified in the received search request is specified with reference to the category database, and the specified category Corresponding to the acquired image ID from the image database, similar image search means for searching the image database associated with the feature data similar to the acquired feature data from the feature database Image data acquisition means for acquiring image data, and the acquired image Characterized in that it comprises an image data transmitting means for transmitting over data to the user terminal.
[0005]
According to the first aspect of the present invention, the feature data representing the feature of each image is stored in the feature database. Therefore, it is possible to determine the similarity between images by comparing these feature data. Therefore, according to the present invention, an image similar to the image designated by the user can be presented. In addition, since it is only necessary to search for feature data belonging to a common category in the feature database, the search target is narrowed down, and the search can be performed quickly.
[0006]
The invention described in claim 2 is the similar image presentation system according to claim 1, and includes a feature server and a product sales server connected to the feature server. The image database, the category database, the search request receiving means, the image data acquisition means, and the image data transmission means. The feature server includes the feature database, the feature data acquisition means, and the similar image search means. It is characterized by providing .
[0007]
The invention described in claim 3 is the similar image presentation system according to claim 1 or 2,
The feature data includes color feature data related to the color of the image, edge feature data related to the edge of the image, and texture feature data related to the texture of the image.
[0008]
Further, the invention described in claim 4 is the similar image presentation system according to any one of claims 1 to 3,
The similar image search means calculates similarity data indicating the height of the similarity between the feature data associated with the acquired image ID and the acquired feature data,
The image data transmitting means transmits a screen for displaying the image data acquired by the image data acquiring means in an order corresponding to the calculated similarity data to the user terminal.
[0009]
The invention described in claim 5 relates to a similar image presentation method for searching for an image similar to an image designated by a user terminal accessed via a network and for browsing on the user terminal. Is.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment when the present invention is applied to an auction system on a network will be described. In Internet auctions, users who solicit bidders on the Internet for a specified period of time and who have offered the highest bid price have the right to purchase the listed products from the seller at that bid price. Is to get. The system of the present embodiment has a function of searching for an image similar to an image designated by the user and presenting it to the user for the product image provided by the seller of the auction product.
[0011]
FIG. 1 is an overall configuration diagram of a system according to the present embodiment. As shown in FIG. 1, the system of this embodiment includes an auction server 1 and a feature server 2. The feature server 2 is connected to the auction server 1. In addition, a user computer 4 is connected to the auction server 1 via a network 3.
[0012]
As will be described later, when the user issues a search request for a similar image while viewing an image of a certain product, the auction server 1 transmits an auction ID unique to the product to the feature server 2. When the feature server 2 receives the auction ID from the auction server 1, the feature server 2 searches for a product image similar to the image corresponding to the auction ID, and transmits the auction ID corresponding to the searched similar image to the auction server 1.
[0013]
As shown in FIG. 1, the auction server 1 is provided with an image database 5 and an auction database 6. The feature server 2 is provided with a feature database 7 and a category database 8.
[0014]
The image database 5 stores image data representing each product that has been exhibited in the auction. The auction database 6 stores auction data including bid prices, the number of bids, the remaining time of the auction, and the like related to each product. The feature database 7 stores feature data representing features of each product image. Furthermore, the category database 8 stores category data indicating the correspondence between the auction ID and the category of the corresponding product.
[0015]
FIG. 2 shows an example of auction data recorded in the auction database 6. As shown in the figure, auction data is recorded in the auction database 6 while being classified into categories. The auction data includes data items such as the auction ID, the URL of the image data of the product image, the bid price in the auction, the number of bids, and the remaining time of the auction. Note that the “current price” in FIG. 2 indicates, for example, a price obtained by adding a bid unit price to the price of the bidder who bids the second highest price.
[0016]
FIG. 3 shows an example of feature data recorded in the feature database 7. As shown in the figure, the feature data is also recorded separately for each product category, similar to the auction data. In the present embodiment, the color, edge (contour), and texture (pattern or pattern) are obtained from the feature data of the feature server 2 described later from the image data of all the products provided by the seller of the products to be exhibited at the auction. Three types of features (pattern) are extracted, and the data is stored in the feature database 7 as feature data.
[0017]
Next, the function of the feature server 2 will be described. The feature server 2 has the following two functions.
(1) A function (feature extraction function) for extracting three types of features, which will be described later, from color, edge (contour), and texture (pattern or pattern) from image data.
(2) Data indicating the high degree of similarity by extracting the auction ID of an image similar to the image of the designated auction ID when a request for similar image retrieval specifying the auction ID is received from the auction server 1 A function of notifying the auction server 1 together (similarity determination function).
Hereinafter, these functions will be briefly described (for details, see, for example, Japanese Patent Laid-Open No. 2000-285141).
[0018]
<Feature extraction function>
There are three types of features extracted by the feature extraction function: product color, edge representing product outline, and texture (pattern or pattern) on the product surface.
[0019]
When extracting the color characteristics of a product image, first, for each pixel included in the image, the color is three-dimensional based on the RGB color system specified by the gradation values of red, green, and blue. To which coordinate in the color vector space is associated. The RGB histogram formed by performing such association for all the pixels is used as the color feature data of the product image.
[0020]
When extracting the feature of the edge of the product image, the entire image data is converted into an array of numbers that is first-order-differed by, for example, the Sobel operator. Generally, in the array region corresponding to the edge of the image, the primary difference value shows a large value. Therefore, it is considered that there is an edge near the array where the primary difference value shows a large value. That is, it can be said that the primary difference value array digitizes the edge features of the image. Therefore, the primary difference value array by the Sobel operator is used as the edge feature data of the product image.
[0021]
When extracting the texture of a product image, the power spectrum obtained by performing Fourier transform on the data obtained by quantifying the brightness of the image is the pattern of the brightness of the image or pattern direction and the size of the image element. It can be said that it describes the distribution of. Therefore, the power spectrum in the array obtained by converting the array of pixels that are the two-dimensional coordinates of the image into a complex plane is used as the texture feature data of the product image.
[0022]
Note that the color feature data, edge feature data, and texture feature data described above are all expressed as vector data.
In the present embodiment, when registering a product to be newly exhibited, feature data is extracted from the image of the product by the feature extraction function of the feature server 2 and recorded in the feature database 7. .
[0023]
<Similarity judgment function>
When there is a similar image search request designating an auction ID from the auction server 1, the feature server 2 first refers to the category data of the category database 8 described above, and from the designated auction ID, the feature database 7 Identify categories. Next, the feature server 2 reads the feature data of the designated auction ID, and calculates the Euclidean distance between this feature data and each feature data belonging to the specified category as color feature data, edge feature data, and Each of the texture feature data is obtained, and these three types of Euclidean distances are added. Here, the Euclidean distance is defined as a distance between vector end points of two feature data that are vector data. Therefore, it is determined that the total similarity is higher in the characteristics of the color, the edge, and the texture as the total of the three types of Euclidean distances is smaller.
[0024]
In the present embodiment, it is determined that the images are similar when the obtained Euclidean distance is smaller than a predetermined reference value. That is, the feature server 2 extracts an auction ID in which the total of the three types of Euclidean distances is smaller than the reference value, and indicates the high degree of similarity with the total value of the three types of Euclidean distances together with the auction ID. Data is transmitted to the auction server 1 as data (hereinafter referred to as similarity data).
It should be noted that not only the above-mentioned Euclidean distance but also a unique distance can be defined in accordance with the type of image feature data to obtain similarity data.
[0025]
Next, the flow of processing from when the user accesses the auction server 1 to when a similar image search request is made until the similar image is displayed on the user computer 4 will be described with reference to FIG.
[0026]
First, when the user computer 4 accesses the auction server 1 (S100), a list of auction product categories is displayed on the user computer 4 from the auction server 1 (S101). When the user computer 4 selects a category (S102), a list of products belonging to the selected category is displayed from the auction server 1 to the user computer 4 (S103). When the user selects a product from the product list (S104), the product image illustrated in FIG. 5 is transmitted from the auction server 1 to the user computer 4 (S105). As shown in FIG. 5, a product image, a “similar search” button 21, and a “bid” button 22 are arranged on the product screen. By clicking the “bid” button 22 on this product screen, it is possible to bid for the product.
[0027]
When the user wants to browse a product whose appearance is similar to the displayed product, the user clicks a “similar thing search” button 21 of the product image. In response to this, a search request for a similar image specifying the auction ID (“K123” in the example of FIG. 5) of the displayed product is transmitted from the user computer 4 to the auction server 1 (S106).
[0028]
In response to the above search request, the auction server 1 transmits a similar image search request for the designated auction ID to the feature server 2 (S107).
[0029]
When the feature server 2 receives the search request from the auction server 1 (S108), the feature server 2 extracts the auction ID of the product image similar to the product image of the designated auction ID by the above-described similarity determination function (S109). Then, the extracted auction ID is transmitted to the auction server 1 together with the similarity data (S110).
[0030]
If the auction ID having similarity data smaller than the reference value is not included in the feature data, the feature server 2 notifies the auction server 1 that there is no similar image (S111).
[0031]
When the auction server 1 receives the auction ID and the similarity data corresponding to the auction ID from the feature server 2 (S112), the auction is received from the auction data of the category specified by the auction ID with reference to the auction database 6. The URL of the image data corresponding to each ID is acquired (S113). Next, the image data of the acquired URL is read with reference to the image database 5 (S114). Then, a search result display screen for displaying the read image together with the original product image in the order of the value of the similarity data in ascending order (that is, in descending order of similarity) is generated and transmitted to the user computer 4 (S115). This screen is displayed on the user computer 4 (S116).
[0032]
When the feature server 2 notifies that there is no similar image, a screen including a message indicating that there is no similar product is transmitted to the user computer 4 (S117).
[0033]
FIG. 6 is a diagram showing an example of a search result display screen displayed on the user computer 4 in S116. In the example shown in the figure, the original product image requested by the user to search for a similar image is arranged at the top of the search result display screen. Below this image, a category name, the total number of similar product images searched, and a list of similar product images are displayed. If you click on each image, you can move to the display screen for each product. As described above, the display order of similar images is the order of high similarity by default, but by clicking “Current Price”, “Number of Bids”, and “Remaining Time” included in the screen, It is possible to sort by price order, number of bids, and remaining time.
[0034]
As described above, according to the system of the present embodiment, the user browses data of other products that are similar in appearance to the displayed products, and refers to the bid amount when the user participates in the auction, or It will be possible to find a product closer to your preference and bid.
[0035]
Further, since the feature data is classified and recorded for each category and the feature server 2 uses the feature data of the corresponding category as a search target for similar images, the search speed can be improved by narrowing down the search target. However, if the feature server 2 has a sufficiently high speed processing capability, the feature data need not necessarily be classified for each category.
[0036]
In the above embodiment, the case where the present invention is applied to an auction system has been described. However, the present invention is not limited to this, and various images are presented to a user such as an electronic mall system. Can be provided to the system.
[0037]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to show the image similar to the image which the user selected with respect to the user terminal accessed via the network.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an overall configuration of a system according to an embodiment.
FIG. 2 is a diagram showing an example of auction data in the present embodiment.
FIG. 3 is a diagram showing an example of feature data in the present embodiment.
FIG. 4 is a diagram showing a flow of processing in the present embodiment.
FIG. 5 is a diagram illustrating an example of a product display screen displayed on a screen of a user computer.
FIG. 6 is a diagram showing an example of a search result display screen displayed on the screen of the user computer.
[Explanation of symbols]
1 image server 2 feature server 3 network 4 user computer 5 image database 6 auction database 7 feature database 8 category database

Claims (5)

A similar image presentation system for configuring a product sales site and searching for an image similar to a product image specified by a user terminal accessed via a network and for browsing on the user terminal,
An image database storing image data of various products sold at the product sales site;
A product database in which product information of the various products is stored for each category;
For each piece of image data stored in the image database, an image ID is associated with feature data representing the feature of the image, and is recorded separately according to the category of the product represented by the image in the product database. Feature database,
A category database for storing category data indicating a correspondence relationship between the image ID and a category of the product in the product database ;
Search request receiving means for receiving a search request for a similar image designating an image ID from a user terminal;
Feature data acquisition means for acquiring feature data associated with the image ID specified in the received search request from the feature database;
The category corresponding to the image ID specified in the received search request is specified with reference to the category database, and the feature data belonging to the specified category is targeted, and the feature data similar to the acquired feature data is supported. Similar image search means for searching the attached image ID from the feature database;
Image data acquisition means for acquiring image data corresponding to the acquired image ID from the image database;
A similar image presentation system comprising: image data transmission means for transmitting the acquired image data to the user terminal. The similar image presentation system according to claim 1,
Comprising a feature server and a product sales server connected to the feature server,
The commodity sales server includes the image database, the category database, the search request receiving unit, the image data obtaining unit, and the image data transmitting unit.
The feature server includes the feature database, the feature data acquisition unit, and the similar image search unit. In the similar image presentation system according to claim 1 or 2,
The similar image presentation system, wherein the feature data includes color feature data relating to an image color, edge feature data relating to an edge of the image, and texture feature data relating to a texture of the image. In the similar image presentation system as described in any one of Claims 1-3,
The similar image search means calculates similarity data indicating the height of the similarity between the feature data associated with the acquired image ID and the acquired feature data,
The image data transmitting unit transmits a screen for displaying the image data acquired by the image data acquiring unit in an order corresponding to the calculated similarity data to the user terminal. . A similar image presentation method for searching for an image similar to an image specified by a user terminal that has accessed a product sales site via a network and for browsing on the user terminal,
A procedure in which a computer receives a search request for a similar image designating an image ID from the user terminal via the network;
For each image data stored in the image database, the image is associated with an image ID and feature data representing the characteristics of the image, and the image in the product database stores product information of various products for each category. Obtaining feature data associated with the image ID specified in the received search request from the feature database recorded separately according to the product category represented by
The computer identifies a category corresponding to the image ID specified in the received search request with reference to a category database that stores category data indicating a correspondence relationship between the image ID and the category of the corresponding product, and the identification Searching for an image ID associated with feature data similar to the acquired feature data from the feature database for feature data belonging to the selected category;
A procedure for acquiring image data corresponding to the acquired image ID from the image database;
A computer comprising a procedure for transmitting the acquired image data to the user terminal via the network.

Images