JP2014191588A - Content search device, content search system, content search method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content search device capable of searching for content similar to a whole design desired by a user, and similar to a partial design.SOLUTION: A content search device includes: a first reception part for receiving, from a user terminal, the information of a first image including a global feature as the overall feature of content and the information of a second image including a local feature as the partial feature of the content as the information of an image of search conditions; a search part for searching for the image of content including a feature similar to the global feature of the first image corresponding to the information of the first image and a feature similar to the local feature of the second image corresponding to the information of the second image from among images corresponding to the information of the images of a plurality of contents in the same category stored in an image storage part; and a first transmission part for transmitting at least either one of the information of the image of the content including the feature similar to the global feature and the information of the image of the content including the feature similar to the local feature to the user terminal.

Description

  The present invention relates to a content search apparatus, a content search system, a content search method, and a program for searching for an image of a content including features similar to a global feature and a local feature included in a search condition image from a plurality of content images. It is about.

  On current e-commerce websites, such as internet shopping websites that use the Internet, images of products (real content) in the same category can be displayed for each manufacturer, for example, and text can be used as a search key It is possible to do.

  For example, when a user wants to buy a wristwatch, a list of manufacturer names is displayed when a wristwatch link on an electronic commerce website is selected by clicking with a mouse or touching a touch panel.

  In this case, if the user wants to buy a wristwatch of a favorite manufacturer, a list of wristwatch images of the manufacturer is displayed by selecting the link of the manufacturer, and the user's favorite wristwatch is searched from the list. be able to. In this case, for example, the display order of the wristwatch image list can be changed according to search conditions such as popularity order, new arrival order, release date order, price order, and order of sale.

  However, depending on the manufacturer and the product, images of thousands of products may be displayed in the list, and it is very difficult to find a user's favorite product from among them.

  Also, in the search for products by text, for example, if the product name, model number, etc. of the product to be purchased are known, it is possible to search for the desired product by inputting it and performing a text search. .

  However, when searching for a product by replacing the overall shape of the product or the overall color of the product with text as a search key, there are problems in the following points (1) and (2) from the viewpoint of user convenience. It was.

(1) Since the design of the product imaged in the head by the user is converted into text and the search is performed using the search key as a search key, the searched product and the design of the product imaged by the user in the head And may be different. For example, if you want to search for a vivid yellow oval wristwatch, even if you search for “yellow” and “oval” in the text, you can search for the wristwatch that the user imagined in your head. I can't.

(2) Regarding the overall shape of the product and the overall color of the product, it is possible to search for the desired product. For example, the shape of the watch, the material of the belt, etc. Even if you want to search for products that have the desired design for details, shapes, colors, textures (patterns, textures, etc.), there is no means to convey that nuance appropriately, You cannot search for textures.

  Here, there are patent documents 1 to 3 as prior art documents relevant to the present invention.

Patent Document 1 relates to an image search apparatus. This document extracts a local feature quantity that is a local feature quantity from an input image, and a local feature similarity that is a similarity between the local feature quantity and a local feature quantity of each image registered in the image feature database. And obtaining a plurality of candidate images based on the calculated local feature similarity, extracting additional features that are additional features from each of the input image and the plurality of candidate images, and each of the plurality of candidate images. The additional feature similarity that is the similarity between the additional feature extracted from the input image and the additional feature extracted from the input image is calculated, and the total similarity is calculated using the local feature similarity and the additional feature similarity. It is described that a predetermined number of candidate images are output as search results in the order of similarity.
The same document also discloses a method of searching for a similar image using the overall feature amount (overall feature amount) of the image, and a method of searching for a similar image using the local feature amount (local feature amount) of the image. Is described.

Patent Document 2 relates to an image search program. In this document, multiple types of feature quantities are extracted from the key image, and whether or not the similarity that matches the human sense of similarity is calculated for each type of feature quantity is determined as the presence or absence of suitability. In addition, it is described that the similarity between the key image and each image to be searched is calculated based on the determination result, and the search result of the image corresponding to the similarity is displayed.
Further, this document exemplifies color layout feature amounts, texture feature amounts, and shape feature amounts as feature amount types.

  Patent Document 3 relates to a similar image search device. This document extracts the shape information and texture information of the subject included in the subject image represented by the input image data, and the shape information of the subject included in the subject image represented by the image data stored in the image database. And the texture information are compared to calculate the similarity between the two subject images, and based on the similarity, the subject similar to the subject image represented by the input image data from the image data stored in the image database It describes that similar image data representing an image is retrieved and retrieval result information representing the retrieval result is output.

JP 2011-2966 A Table 2004/47026 JP-A-2005-65728

  An object of the present invention is to provide a content search apparatus and a content search system that can solve the problems of the prior art and can search for content similar to the overall design desired by the user and similar to a partial design. Another object is to provide a content search method and program.

In order to achieve the above object, the present invention is a content search device that is connected to one or more user terminals via a network and that searches for images of content similar to the search condition image from among a plurality of content images. Because
An image storage unit for storing image information of a plurality of contents in the same category;
From the user terminal, as information on the image of the search condition, information on the first image including the global feature that is the overall feature of the content, and information on the second image including the local feature that is a partial feature of the content. A first receiving unit for receiving;
Similar to the global feature of the first image corresponding to the information of the first image received by the first receiver out of the images corresponding to the information of the images of the plurality of contents of the same category stored in the image storage unit A search unit for searching for an image of content including a feature and including a feature similar to the local feature of the second image corresponding to the information of the second image received by the first receiving unit;
At least one of content image information including features similar to the global features searched by the search unit and content image information including features similar to the local features searched by the search unit is transmitted to the user terminal. A content search apparatus including a first transmission unit is provided.

  Here, the global features preferably include at least one of an overall shape of the content, an overall color of the content, and an overall texture of the content.

  In addition, the local features include content partial shape, content partial size, content partial number, content partial position, content partial color, and content partial texture. It is preferable to include at least one.

Further, the present invention provides the content search device according to any one of the above,
One or more user terminals connected to the content search device via a network,
The user terminal transmits information on the image of the search condition to the content search device, and the content including features similar to the global feature and the local feature included in the image of the search condition corresponding to the information on the image of the search condition from the content search device The content search system is characterized in that it receives the image information and displays an image corresponding to the received content image information.

Here, the user terminal is
A second receiving unit for receiving content image information from the content search device via a network;
A display unit for displaying an image corresponding to information on an image of the content received by the second receiving unit;
An input unit for inputting an instruction for designating a global feature or a local feature for an image of content displayed on the display unit;
A second transmission unit that transmits the information of the first image including the global feature or the information of the second image including the local feature, which is input via the input unit, to the content search device as the image information of the search condition;
It is preferable to include a control unit that controls operations of the second reception unit, the display unit, the input unit, and the second transmission unit.

The present invention is also a content search method for performing image search for a content image similar to an image of a search condition from a plurality of content images in a content search device connected to one or more user terminals via a network. There,
The first receiving unit of the content search device receives, from the user terminal, information on the first image including the global features that are the overall features of the content as information on the search condition images.
The search unit of the content search apparatus includes a first corresponding to the first image information received by the first receiving unit from among the images corresponding to the information of the plurality of content images of the same category stored in the image storage unit. Search for content images that contain features similar to the global features of an image,
The first transmission unit of the content search device transmits information on content images including features similar to the global features searched by the search unit to the user terminal,
The first receiving unit receives, from the user terminal, information on the second image including local features that are partial features of the content as image information of the search condition,
Content including a feature similar to the local feature of the second image corresponding to the information of the second image received by the first receiver from among the images of the content including features similar to the global feature searched by the search unit Search for images of
A content search method is provided in which a first transmission unit transmits information of an image of a content including a feature similar to the local feature searched by the search unit to a user terminal.

Here, the second receiving unit of the user terminal receives, from the content search device, information on the representative image of each category of content in which images of a plurality of content in the same category are classified according to global features,
The control unit of the user terminal causes the display unit to display a representative image corresponding to the information of the representative image of each category of content received by the second receiving unit,
Information on the representative image of one or more contents specified by an instruction input via the input unit of the user terminal from the representative images of the contents of each category displayed on the display unit by the second transmission unit of the user terminal To the content search device as information on the first image,
A second receiving unit receives information on a content image including a feature similar to the global feature from the content search device;
The control unit causes the display unit to display an image corresponding to information on the content image including the feature similar to the global feature received by the second receiving unit,
The second transmission unit transmits the content image including one or more local features specified by the instruction input via the input unit from the content image including the feature similar to the global feature displayed on the display unit. Information is transmitted to the content search device as information of the second image,
A second receiving unit receives information on an image of content including a feature similar to a local feature from the content search device;
Preferably, the control unit causes the display unit to display an image corresponding to information on the content image including the feature similar to the local feature received by the second receiving unit.

In addition, the second receiving unit receives information on the representative image of the content of each classification classified by the global feature from the content search device,
The control unit displays the representative image corresponding to the representative image information of the content of each category on the display unit,
The second transmission unit transmits the information of the representative image of one or more contents as the information of the first image to the content search device,
The first receiving unit receives the first image information from the user terminal;
The search unit searches for an image of content including features similar to the global features of the first image corresponding to the information of the first image, and performs a refinement search by repeating the search twice or more while changing the global features,
It is preferable that the first transmission unit transmits information on the content image including the feature similar to the global feature subjected to the narrowing search by the search unit to the user terminal.

Also, the first receiving unit receives from the user terminal information on the second image including two or more local features as data on the search condition, and data on the time required for specifying each of the two or more local features. Receive
Based on the time data, the search unit changes the weighting of each of the two or more local features according to the time required to specify each of the two or more local features, and corresponds to the second image information. It is preferable to search for images of content that contain features similar to two or more local features of the image.

In addition, the control unit displays an image of one content selected by an instruction input via the input unit from images corresponding to content image information including features similar to the global feature on the display unit. Let
The second transmission unit obtains information on the content image including one or more local features specified by the instruction input via the input unit from the content image displayed on the display unit. Information is sent to the content search device,
The control unit displays an image corresponding to the content image information including the feature similar to the local feature received by the second receiving unit, together with the content image specified by the instruction input through the input unit. It is preferable to display on the part.

  In addition, the control unit displays a predetermined number of content images on the display unit from among the images corresponding to the information on the content image including features similar to the global features, and the predetermined number of images displayed on the display unit An instruction for designating a content image including a local feature of the user's preference is input from the content image via an input unit, and the next predetermined number of content images are displayed. When the instruction is input, the image of the content including the feature similar to the local feature of the user's preference is preferentially selected from the images corresponding to the information of the content image including the feature similar to the global feature. It is preferable to display a number of content images on the display unit.

  In addition, the control unit causes the display unit to display a predetermined number of content images from among images corresponding to the content image information including features similar to the global features, and the predetermined number of contents displayed on the display unit. In response to an instruction input via the input unit, an instruction to specify an image of content that the user dislikes is input, and an instruction to display the next predetermined number of content images is input. And displaying the next predetermined number of content images by excluding content images similar to the content image disliked by the user from images corresponding to content image information including features similar to the global features It is preferable to display on the part.

  The present invention also provides a program for causing a computer to execute each step of the content search method described above.

  The present invention also provides a computer-readable recording medium on which a program for causing a computer to execute each step of the content search method described above is recorded.

In the present invention, first, an image including a feature similar to the global feature included in the first image is searched, and then an image including a feature similar to the local feature included in the second image is searched.
According to the present invention, by performing an image search using the first image and the second image, even a design that cannot be expressed well by text can be used as a search key, and content having a desired design can be easily obtained. You can search. Also, by searching by global features and local features, it is possible to efficiently search for content that has a design similar to the overall design of content in the user's head and similar to a partial design. it can.

It is a conceptual diagram of one Embodiment showing the structure of the content search system of this invention. It is a block diagram showing the structure of the content search apparatus shown in FIG. It is a block diagram showing the structure of the user terminal shown in FIG. It is a flowchart showing operation | movement of the content search system shown in FIGS. It is a flowchart showing schematic operation | movement of the specific example of a content search system. It is a conceptual diagram showing the screen which designates the case shape of a wristwatch as a global feature. It is a conceptual diagram showing a mode that a * mark is selected and changed into a * mark. It is a conceptual diagram showing the screen which designates the dial color of a wristwatch as a global feature. It is a conceptual diagram showing the display screen of the image of the wristwatch containing the feature similar to the global feature. It is a conceptual diagram showing the screen which designates a local feature in the image of the selected wristwatch. It is a conceptual diagram showing the display screen of the image of the wristwatch containing the feature similar to the global feature after the local feature is designated.

  Hereinafter, a content search device, a content search system, a content search method, and a program according to the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

  FIG. 1 is a conceptual diagram of an embodiment showing a configuration of a content search system of the present invention. The content search system 10 shown in the figure is a product search system, which displays an image of a product (real content) as an example of content, and a user operation (designation of global features and local features) on the displayed product image. ) To search for and display an image of a product having a design desired by the user. The content search system 10 includes a product search device 12 and one or more user terminals 14 connected to the product search device 12 via a network 16 such as the Internet.

  The product search device (content search device) 12 is a server that provides a website for electronic commerce that sells products such as watches and socks, for example, and a search received from the user terminal 14 from a plurality of product images. The image of the product including features similar to the global feature and the local feature included in the image of the search condition corresponding to the information of the image of the condition is image-searched, and the information of the image of the searched product is transmitted to the user terminal 14. As shown in FIG. 2, the product search device 12 includes an image storage unit 18, a first reception unit 20, a search unit 22, and a first transmission unit 24.

  Here, the image information includes, for example, image data (digital data) corresponding to the image or ID (identification) information of the image. That is, when image information is transmitted and received between the product search device 12 and the user terminal 14, the image data itself corresponding to the image may be transmitted or received, or the image ID information may be transmitted and received instead of the image data itself. Also good. It is desirable because the amount of data transmitted and received between the product search device 12 and the user terminal 14 can be reduced by transmitting and receiving image ID information.

  The first receiving unit 20 receives information about the first image including global features, which are the overall features of the product, and partial features of the product as information about the search condition image from the user terminal 14 via the network 16. Various data including the information of the second image including the local feature is received.

Here, the global features include at least one of the overall shape of the product, the overall color of the product, and the overall texture of the product. Local features include product partial shape, product partial size, product partial quantity, product partial location, product partial color and product partial texture (pattern, texture, etc. ).
In addition, what kind of characteristic is included as a global characteristic and a local characteristic is arbitrary, and things other than the above can also be included.

  The image storage unit 18 stores information on images of a plurality of products in the same category for each product category, such as a wristwatch image and a sock image.

  The search unit 22 selects the first image corresponding to the first image information received by the first receiving unit 20 from the images corresponding to the image information of a plurality of products of the same category stored in the image storage unit 18. Search for images of products that contain features similar to the global features of the image. Further, the search unit 22 is similar to the local feature of the second image corresponding to the information of the second image received by the first receiving unit 20 among the images of the products including features similar to the searched global feature. Search for product images that contain features. That is, the search unit 22 searches for an image of a product that includes a feature similar to the global feature of the first image and includes a feature similar to the local feature of the second image.

  The first transmission unit 24 includes information on product images including features similar to the global features searched by the search unit 22 and information on products images including features similar to the local features searched by the search unit 22. Various types of data including at least one of the above are transmitted to the user terminal 14 via the network 16.

  Subsequently, the user terminal 14 is a communication terminal such as a PC (personal computer) owned by a user who uses the content search system 10, and information on search condition images (information on the first image and information on the second image). To the product search device 12, and receives from the product search device 12 information on the product image including features similar to the global feature and the local feature included in the search condition image corresponding to the search condition image information, An image corresponding to the received product image information is displayed. As shown in FIG. 3, the user terminal 14 includes a second reception unit 26, a display unit 28, an input unit 30, a second transmission unit 32, and a control unit 34.

  The second receiving unit 26 receives various information including product image information including features similar to global features, product image information including features similar to local features, and the like from the product search device 12 via the network 16. Receive data.

  The display unit 28 displays an image corresponding to information on the product image received by the second receiving unit 26 under the control of the control unit 34.

  The input unit 30 is used by the user to input various instructions including instructions for specifying global features or local features to the product image displayed on the display unit 28. As the input unit 30, a mouse, a keyboard, a touch panel, or the like can be used.

  The second transmission unit 32 uses the information of the first image including the global feature or the information of the second image including the local feature input via the input unit 30 as information on the image of the search condition via the network 16. To the product search device 12.

  The control unit 34 controls operations of the second reception unit 26, the display unit 28, the input unit 30, and the second transmission unit 32.

  Next, the operation of the content search system 10 shown in FIGS. 1 to 3 will be described with reference to the flowchart shown in FIG.

  When a user searches for a product with a desired design on an electronic commerce website, the user clicks with the mouse as the input unit 30 or touches the touch panel on the user terminal 14 to indicate the category of the product. Enter instructions to select the link.

When the user inputs an instruction to select a link indicating a product category, the second receiving unit 26 in the user terminal 14 receives a global image of a plurality of products of the same category from the product search device 12 via the network 16. Information on representative images of products of each category classified by the feature is received (step S1).
And the control part 34 displays the representative image corresponding to the information of the representative image of the goods of each classification received by the 2nd receiving part 26 on the display part 28 (step S2).

  Subsequently, the user inputs an instruction for designating a representative image of one or more products including a desired global feature from the representative images of each category of products displayed on the display unit 28 via the input unit 30. .

  When the user inputs an instruction to specify a representative image of a product, the second transmission unit 32 in the user terminal 14 represents one or more product representatives specified by the instruction input via the input unit 30 of the user terminal 14. The image information is transmitted to the product search device 12 via the network 16 as the first image information (step S3).

When the information on the first image is transmitted from the user terminal 14 to the product search device 12, in the product search device 12, the first receiving unit 20 includes a global feature as information on the search condition image from the user terminal 14. Information on the first image is received (step S4).
Subsequently, the search unit 22 corresponds to the information of the first image received by the first receiving unit 20 from the images corresponding to the image information of the plurality of products of the same category stored in the image storage unit 18. A product image including a feature similar to the global feature of the first image is retrieved (step S5).
And the 1st transmission part 24 transmits the information of the image of the goods containing the feature similar to the global feature searched by the search part 22 to the user terminal 14 via the network 16 (step S6).

  Here, one or more global features can be used to perform an image search using a global feature only once, or two or more global features can be sequentially changed to repeat an image search using a global feature two or more times. You can also.

  When performing an image search using a global feature only once, an image search is performed using, for example, the overall shape of the product, the overall color of the product, the overall texture of the product, or a combination thereof as the global feature. .

When performing an image search using the global shape of the product, the overall color of the product, or the overall texture of the product, the search unit 22 is similar to the global feature of the first image. The image of the product including the feature is searched.
When the image search is performed using a combination of the overall shape of the product, the overall color of the product, and the overall texture of the product as the global feature, the search unit 22 is similar to the global feature of the first image combination. A refined search is performed by simultaneously searching for images of products including the feature.
When the search by the global feature is completed, the first transmission unit 24 transmits information on the image of the product including the feature similar to the global feature searched by the search unit 22 to the user terminal 14.

When the search by the global feature is performed twice or more, the narrowing search is performed by performing the search twice or more step by step while sequentially changing the two or more global features.
In the case of the present embodiment, as the global feature, for example, in order of the overall shape of the product and the overall color of the product, or in the order of the overall color of the product and the overall shape of the product, stepwise 2 Perform a refined search by searching once.

For example, in the case of performing a refinement search step by step in the order of the overall shape of the product and the overall color of the product, the second receiving unit 26 receives the overall shape of the product from the product search device 12 in the user terminal 14. The control unit 34 displays the representative image corresponding to the information of the representative image of the product classified in each category on the display unit 28, and the second transmission unit 32 The information on the representative image of one or more products specified by the instruction input via the input unit 30 of the user terminal 14 is transmitted to the product search device 12 as the first image information.
In the product search device 12, the first receiving unit 20 receives the information of the first image from the user terminal 14, and the search unit 22 has a feature similar to the global feature of the first image corresponding to the information of the first image. Search for images of products that contain them.
Subsequently, the overall shape of the product is changed to the overall color of the product, and the above operation is repeated to perform a narrowing search.
When the refinement search is completed for the global feature, the first transmission unit 24 transmits information about the product image including the feature similar to the global feature subjected to the refinement search by the search unit 22 to the user terminal 14.

Subsequently, when the information on the image of the product including the feature similar to the global feature is transmitted from the product search device 12 to the user terminal 14, the second receiving unit 26 receives the network information from the product search device 12 in the user terminal 14. The image information of the product including the feature similar to the global feature is received via 16 (step S7).
Then, the control unit 34 causes the display unit 28 to display an image corresponding to information on the product image including features similar to the global features received by the second receiving unit 26 (step S8). At this time, the image of the actual product is displayed on the display unit 28 for the first time.

  Subsequently, the user selects an image of one product from images of products including features similar to the global feature displayed on the display unit 28, and one or more local features are selected for the selected product image. An instruction to specify is input via the input unit 30.

  When the user inputs an instruction to specify a local feature, in the user terminal 14, the second transmission unit 32 includes one or more local features specified by the instruction input via the input unit 30 of the user terminal 14 The image information is transmitted as the second image information to the product search device 12 via the network 16 (step S9).

When the information on the second image is transmitted from the user terminal 14 to the product search device 12, the first receiving unit 20 in the product search device 12 receives information about the search condition image from the user terminal 14 via the network 16. As a result, the information of the second image including the local feature which is a partial feature of the product is received (step S10).
Subsequently, the search unit 22 resembles the local feature of the second image corresponding to the information of the second image received by the first receiver 20 from among the images of the product including features similar to the searched global feature. A product image including the feature is searched for (step S11).
And the 1st transmission part 24 transmits the information of the image of the goods containing the feature similar to the local feature searched by the search part 22 to the user terminal 14 via the network 16 (step S12).

  Here, it is also possible to perform a search based on local features by changing the weighting of each of the two or more local features according to the time required for specifying each of the two or more local features.

In this case, in the product search device 12, the first receiving unit 20 receives information about an image including two or more local features and information about two or more local features from the user terminal 14 as image information of the search condition. Receive data for the time required for specification.
Then, the search unit 22 resembles two or more local features by changing the weighting of each of the two or more local features according to the time required for specifying each of the two or more local features based on the time data. Search for images of products that contain the feature.
When the user designates a favorite local feature, it is considered that the time required for the designation becomes longer. Accordingly, the search unit 22 searches for an image of the product with a higher weight as the time required for specifying the local feature becomes longer.
For example, when the time required for specifying each of the two local features is 1 second and 5 seconds, the search unit 22 searches for the product image with a weight of 1: 5. As a result, a similar image is searched when the time required for specifying the local feature is 5 seconds rather than 1 second.

Subsequently, when information on the product image including the feature similar to the local feature is transmitted from the product search device 12 to the user terminal 14, the second receiving unit 26 transmits the network information from the product search device 12 to the user terminal 14. The image information of the product including the feature similar to the local feature is received via 16 (step S13).
And the control part 34 displays the image corresponding to the information of the image of the goods containing the feature similar to the local feature received by the 2nd receiving part 26 on the display part 28 (step S14).

  The user inputs an instruction for designating an image of a desired product from the product image including features similar to the local feature displayed on the display unit 28 via the input unit 30. Thereby, the user can search for a product having a desired design.

  Next, the operation of the content search system 10 will be described in detail by giving an example of searching for a wristwatch with a desired design.

  In this specific example, as shown in FIG. 5, first, a feature similar to the first global feature representing the overall shape of the wristwatch included in the representative image designated from the representative images of the plurality of wristwatches is included. A rough aperture search for narrowing the image is performed (step S15), and then an intermediate aperture search for narrowing an image including a feature similar to the second global feature representing the overall color of the wristwatch is performed (step S16). Then, a final aperture search is performed to narrow down an image including features similar to the specified local features for the image selected from the images subjected to the intermediate aperture search (step S17) (step S18).

Similarly, when the user inputs an instruction to select a link indicating a wristwatch category on the electronic commerce website, the case of each case shape received from the product search device 12 as shown in FIG. A representative image corresponding to the information of the representative image of the wristwatch is displayed.
In the figure, representative images of wrist watches of each case shape are shown in a round shape, a square shape, a rectangular shape, an oval shape, a tonneau shape, an octagon shape, a cushion shape, and a cylinder shape.

In this embodiment, up to eight wristwatch images are displayed simultaneously. When the user inputs an instruction to select the “next 8” button, representative images of the next 8 wristwatches having different case shapes are sequentially displayed.
Note that it is arbitrary how many wristwatch images are displayed simultaneously. For example, representative images of all case-shaped watches can be displayed, and the “next eight” buttons can be dispensed with.

  Subsequently, the user designates one or more wristwatch representative images including a desired case shape from the representative images of the wristwatches of each case shape shown in FIG. 6, and then instructs to select the “Next” button. input.

In this embodiment, as shown in FIG. 7, the case shape is round as shown in FIG. 6 by selecting the ☆ mark placed at the upper left of the representative image of the wristwatch and changing it to a red ★ mark. Assume that a representative image of an oval wristwatch is designated.
That is, the red ★ mark arranged at the upper left of the representative image of the wristwatch represents the representative image of the wristwatch including the case shape desired by the user.
If the red ★ mark is selected again, it can be returned to the ☆ mark.

  Here, various designation methods including a conventionally known designation method can be adopted as a method for designating a representative image of a case shape desired by the user from the representative images of a plurality of watches.

  When an instruction to select the “Next” button is input, information on a representative image of a wrist watch having a round shape and an oval shape is transmitted from the user terminal 14 to the product search device 12 as information on the first image. The

Subsequently, on the user terminal 14, as shown in FIG. 8, a representative image corresponding to the information on the representative image of each dial color watch received from the product search device 12 is displayed.
In the drawing, white, black, red, orange, yellow, green, blue, and purple are shown as representative images of wristwatches of each dial color.

Similarly, when an instruction to select the “next 8” button is input, representative images of the next 8 wristwatches having different dial colors are sequentially displayed.
When an instruction to select the “return” button is input, the screen returns to a screen showing a representative image of each case-shaped wristwatch shown in FIG. In this case, the user can start over from selecting a representative image of a wristwatch including a desired case shape.

  Subsequently, the user designates one or more representative images of wristwatches including a desired dial color from the representative images of the wristwatches of each dial color shown in FIG. 8, and then selects the “Next” button. Enter instructions.

Similarly, it is assumed that the representative image of the wristwatch whose dial color is white and black is designated by selecting the ☆ mark arranged at the upper left of the representative image of the wristwatch and changing it to a red ★ mark.
That is, the red ★ mark placed at the upper left of the wristwatch representative image indicates that the wristwatch representative image includes the user's desired dial color.

  When an instruction to select the “Next” button is input, information on the representative image of the watch whose dial color is white and black is transmitted from the user terminal 14 to the product search device 12 as information on the first image. .

  When the information on the first image is transmitted from the user terminal 14 to the product search device 12, the user is selected from the images corresponding to the plurality of wristwatch image information stored in the image storage unit 18 in the product search device 12. A wristwatch image including features similar to the global features of the first image corresponding to the first image information received from the terminal 14, that is, the case shape is round or oval, and the dial color is white or black The images of the wristwatch are retrieved. Then, information on the wristwatch image including features similar to the searched global features is transmitted from the product search device 12 to the user terminal 14.

  Each time the product search unit 12 receives information on the first image that designates the case shape and the dial color, the product search unit 12 may perform a search in a stepwise manner to narrow down the search. A narrow search may be performed by performing a search simultaneously after receiving information.

When information on a wristwatch image including features similar to global features is transmitted from the product search device 12 to the user terminal 14, the user terminal 14 displays the global features received from the product search device 12 as shown in FIG. An image corresponding to the information of the watch image including similar features is displayed.
In the lower part of each watch image shown in the figure, the manufacturer and model number of the watch are described.

Similarly, when an instruction to select the “next eight” button is input, the next eight wristwatch images among the wristwatch images including features similar to the global feature are sequentially displayed.
When an instruction to select the “return” button is input, the screen returns to a screen showing a representative image of each dial color watch shown in FIG. In this case, the user can start over from selecting a representative image of a wristwatch having a desired dial color.

  Subsequently, the user selects one watch image from the watch images including features similar to the global features shown in FIG.

In the present embodiment, it is assumed that an instruction for designating the upper left wristwatch image is input from the eight wristwatch images shown in FIG. In response to this, as shown in the upper part of FIG. 10, an image of the selected upper left wristwatch and an explanation about the wristwatch are displayed.
In this case, it is desirable to enlarge and display the selected wristwatch image so that the user can easily input an instruction to specify local features on the wristwatch image shown in FIG.

Subsequently, the user inputs an instruction for designating one or more local features by selecting a portion to be a desired local feature on the wristwatch image shown in FIG.
In the present embodiment, by tracing and surrounding the left curved portion of the case and the right crown portion (indicated by the dotted line in the figure) with the mouse, the wristwatch image shown in FIG. It is assumed that an instruction designated as a local feature has been input.

  When an instruction to specify a local feature is input, information on a wristwatch image including one or more local features is transmitted from the user terminal 14 to the product search device 12 as second image information.

  When the information on the second image is transmitted from the user terminal 14 to the product search device 12, the product search device 12 receives the second image received from the user terminal 14 from the wristwatch images including features similar to the global features described above. A watch image is retrieved that includes features similar to the local features of the second image corresponding to the information of the two images, ie, the left curve portion of the case and the right crown portion. Then, the information on the watch image including the feature similar to the searched local feature is transmitted from the product search device 12 to the user terminal 14.

  Here, when searching by local feature, it is optional whether to use the shape, size, number, position, color, or texture of the part specified as the local feature, and how to specify it. Can be set to

  Subsequently, when information on a wristwatch image including a feature similar to a local feature is transmitted from the product search device 12 to the user terminal 14, a feature similar to the local feature received from the product search device 12 is displayed on the user terminal 14. The image corresponding to the information of the image of the wristwatch included is displayed in real time in the lower part together with the image of the wristwatch shown in FIG.

  The user inputs an instruction for designating a desired wristwatch image from the wristwatch image including features similar to the local features shown in the lower part of FIG. Thereby, the user can search for a wristwatch having a desired design.

  Similarly, when an instruction to select the “return” button is input on the screen of FIG. 10, the screen returns to the screen representing the image corresponding to the information of the wristwatch image including the feature similar to the global feature shown in FIG. 9. . In this case, the user can start over from selecting a desired watch image.

When an instruction to select the “return” button is input and the screen returns to a screen representing an image corresponding to information on a watch image including a feature similar to the global feature, the local feature is designated as shown in FIG. In the upper left wristwatch image, a portion (indicated by a dotted line in the figure) designated as a local feature is displayed. Further, the ☆ mark arranged at the upper left of the watch image in which the local feature is designated is changed to a red ★ mark.
That is, the red ★ mark represents a wristwatch image including a user's favorite local feature.

  In addition, instead of automatically changing the ☆ mark placed at the upper left of the watch image with the local feature specified to the ★ mark, the user inputs an instruction to select the ☆ mark to change to the red ★ mark. It is also possible to change.

  An instruction for designating a wristwatch image including a local feature of a user's preference, as indicated by a red star mark, according to an instruction input via the input unit 30 from the eight wristwatch images shown in FIG. Is input, and an instruction to select the “next 8” button is input, the control unit 34 selects the local feature of the user's preference from the watch image including the feature similar to the global feature. The following eight wristwatch images are displayed on the display unit 28 with priority given to wristwatch images including similar features.

  As a result, the user can more easily select a watch image including a desired local feature with a smaller number of clicks and a smaller number of taps.

In addition, on the screen representing the watch image including features similar to the global features shown in FIG. 11, the user can change to the blue star mark by inputting an instruction to select the star mark. In the figure, the ☆ marks on the watches of D company DDD4 and G company GGG7 are changed to blue ★ marks. The blue ★ mark indicates that the image is a watch that the user dislikes.
It should be noted that by repeating the selection of the ☆ mark, it can be changed to a red ★ mark, changed to a blue ★ mark, or returned to the ☆ mark.

  From the eight wristwatch images shown in FIG. 11, an instruction to specify a wristwatch image that the user dislikes is input as indicated by a blue star mark according to an instruction input via the input unit 30. When an instruction to select the “next 8” button is input, the control unit 34 displays a wristwatch similar to the wristwatch image that the user dislikes from among the wristwatch images including features similar to the global features. And the next eight wristwatch images are displayed on the display unit 28.

  As a result, the user can more easily select a watch image including a desired local feature with a smaller number of clicks and a smaller number of taps.

As described above, in the present invention, first, an image including a feature similar to the global feature included in the first image corresponding to the information of the first image is searched, and subsequently, the information of the second image is supported. An image including a feature similar to the local feature included in the second image is searched.
By performing an image search using the first image and the second image, even a design that cannot be expressed well by text can be used as a search key, and a product having a desired design can be easily searched. In addition, by searching for each of the global features and local features, it is possible to efficiently search for products having a design similar to the overall design of the product in the user's head and similar to a partial design. it can.

  In addition, although the wristwatch was mentioned as an example as a specific example of the product, the present invention is not limited to this, and can be similarly applied to other categories of products such as socks and clothes. In addition, the present invention is not limited to searching for actual products (real contents), and can be used for searching various digital image data itself (digital contents) including images, pictures, templates, and the like. Is possible. In the present invention, both real content and digital content are expressed as content.

  The product search method of the present invention can be realized as a program for causing a computer to execute each step. Moreover, it is good also as a computer-readable recording medium with which this program was recorded.

The present invention is basically as described above.
Although the present invention has been described in detail above, the present invention is not limited to the above-described embodiment, and it is needless to say that various improvements and modifications may be made without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 Content search system 12 Product search apparatus 14 User terminal 16 Network 18 Image storage part 20 1st receiving part 22 Search part 24 1st transmission part 26 2nd receiving part 28 Display part 30 Input part 32 2nd transmission part 34 Control part

Claims (14)

A content search apparatus that is connected to one or more user terminals via a network, and that searches for an image of content similar to an image of a search condition from a plurality of content images,
An image storage unit for storing image information of a plurality of contents in the same category;
From the user terminal, as information on the image of the search condition, information on a first image including a global feature that is an overall feature of the content and a second feature including a local feature that is a partial feature of the content A first receiver for receiving image information;
The global feature of the first image corresponding to the information of the first image received by the first receiver out of the images corresponding to the information of the images of the plurality of contents of the same category stored in the image storage unit. A search unit that searches for an image of content that includes similar features and includes features similar to the local features of the second image corresponding to the information of the second image received by the first receiving unit;
At least one of content image information including features similar to the global features searched by the search unit and content image information including features similar to the local features searched by the search unit is used as the user terminal. A content search apparatus comprising: a first transmission unit that transmits data to a first transmission unit.   The content search apparatus according to claim 1, wherein the global feature includes at least one of an overall shape of the content, an overall color of the content, and an overall texture of the content.   The local features include a partial shape of the content, a partial size of the content, a partial number of the content, a partial position of the content, a partial color of the content, and a partial content of the content. The content search device according to claim 1, comprising at least one of various textures. The content search device according to any one of claims 1 to 3,
One or more user terminals connected to the content search device via the network,
The user terminal transmits information on the search condition image to the content search device, and is similar to the global feature and local feature included in the search condition image corresponding to the search condition image information from the content search device. A content search system for receiving content image information including the feature and displaying an image corresponding to the received content image information. The user terminal is
A second receiving unit that receives content image information from the content search device via the network;
A display unit for displaying an image corresponding to information on an image of the content received by the second receiving unit;
An input unit for inputting an instruction for designating the global feature or the local feature with respect to the image of the content displayed on the display unit;
A second transmission unit configured to transmit the information on the first image including the global feature or the information on the second image including the local feature, which is input via the input unit, to the content search device as the image information of the search condition; When,
The content search system according to claim 4, further comprising: a control unit that controls operations of the second reception unit, the display unit, the input unit, and the second transmission unit. In a content search apparatus connected to one or more user terminals via a network, a content search method for searching for an image of content similar to an image of a search condition from a plurality of content images,
A first receiving unit of the content search device receives, from the user terminal, information of a first image including a global feature that is an overall feature of the content as image information of the search condition;
The search unit of the content search device uses the first image information received by the first reception unit from among the images corresponding to the image information of a plurality of contents of the same category stored in the image storage unit. Search for images of content that contain features similar to the global features of the corresponding first image,
A first transmission unit of the content search device transmits information of a content image including features similar to the global features searched by the search unit to the user terminal;
The first receiving unit receives, from the user terminal, information on a second image including local features that are partial features of the content as image information on the search condition,
A feature similar to the local feature of the second image corresponding to the information of the second image received by the first receiver from among the content images containing features similar to the searched global feature. Search for content images that contain
The content search method, wherein the first transmission unit transmits information of a content image including a feature similar to the local feature searched by the search unit to the user terminal. A second receiving unit of the user terminal receives, from the content search device, information on a representative image of each category of content in which images of a plurality of content in the same category are classified according to the global feature;
The control unit of the user terminal causes the display unit to display a representative image corresponding to the information of the representative image of each category of content received by the second receiving unit;
The second transmission unit of the user terminal is a representative of one or more contents specified by an instruction input via the input unit of the user terminal from the representative images of the contents of each category displayed on the display unit Transmitting image information as the first image information to the content search device;
The second receiving unit receives information of an image of content including a feature similar to the global feature from the content search device;
The control unit causes the display unit to display an image corresponding to information on a content image including a feature similar to the global feature received by the second receiving unit;
Content including one or more local features specified by an instruction input via the input unit from an image of content including features similar to the global features displayed on the display unit by the second transmission unit Information of the image is transmitted to the content search device as information of the second image,
The second receiving unit receives information on an image of content including a feature similar to the local feature from the content search device;
The content search method according to claim 6, wherein the control unit causes the display unit to display an image corresponding to content image information including a feature similar to the local feature received by the second reception unit. The second receiving unit receives, from the content search device, information on a representative image of each category of content classified by the global feature;
The control unit causes the display unit to display a representative image corresponding to information of a representative image of the content of each category;
The second transmission unit transmits information of the representative image of the one or more contents as the information of the first image to the content search device;
The first receiving unit receives the information of the first image from the user terminal;
The search unit searches for an image of content including a feature similar to the global feature of the first image corresponding to the information of the first image, and performs a refinement search by repeating the search twice or more while changing the global feature. ,
The content search method according to claim 7, wherein the first transmission unit transmits information about an image of a content including a feature similar to the global feature subjected to the narrowing search by the search unit to the user terminal. The first receiving unit required from the user terminal to specify each of the second image information including the two or more local features and the two or more local features as the image information of the search condition. Receive time data,
Based on the time data, the search unit changes the weighting of each of the two or more local features according to the time required for specifying each of the two or more local features, and information on the second image The content search method according to claim 7 or 8, wherein an image of content including features similar to two or more local features of the second image corresponding to is searched. The display unit displays one content image selected by an instruction input via the input unit from among images corresponding to content image information including features similar to the global feature. Displayed on the
The second transmission unit receives information on the content image including one or more local features specified by an instruction input via the input unit from the content image displayed on the display unit. As information of the second image, it is transmitted to the content search device,
The image of the content specified by the instruction input via the input unit, the image corresponding to the information of the image of the content including the feature similar to the local feature received by the second receiving unit by the control unit And the content search method of any one of Claims 7-9 displayed on the said display part.   The control unit causes the display unit to display a predetermined number of content images from among images corresponding to content image information including features similar to the global features, and the predetermined number displayed on the display unit. In accordance with an instruction input via the input unit, an instruction for designating an image of the content including the user's favorite local feature is input, and the next predetermined number of content images are When an instruction to display is input, priority is given to an image of content including a feature similar to the user's favorite local feature among images corresponding to information on the content image including a feature similar to the global feature. The content search method according to claim 7, wherein an image of the next predetermined number of contents is displayed on the display unit.   The control unit causes the display unit to display a predetermined number of content images from among images corresponding to content image information including features similar to the global features, and the predetermined number displayed on the display unit. In accordance with an instruction input via the input unit, an instruction for designating an image of content that the user dislikes is input, and an instruction to display the next predetermined number of content images is displayed. When input, the image of the content similar to the image of the content disliked by the user is excluded from the images corresponding to the information of the content image including the feature similar to the global feature, and the following predetermined number of images are excluded. The content search method according to claim 7, wherein an image of content is displayed on the display unit.   The program for making a computer perform each step of the content search method of any one of Claims 8-12.   A computer-readable recording medium in which a program for causing a computer to execute each step of the content search method according to any one of claims 8 to 12 is recorded.

Images