JP2006031200A - Commodity retrieval system and method, and terminal device, server device and program used therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To facilitate update management of commodity information while suppressing increase in capacity of a database in a commodity retrieval system using pattern matching by image patterns. <P>SOLUTION: A user terminal 1 transmits an image pattern of a desired commodity to an image processing server 2 through a communication network. The server 2 compares this image pattern with preliminarily stored standard patterns of commodities to retrieve similar/matched commodities. Commodity recognition information for specifying the retrieved commodities is transmitted to shop servers 3 which are dispersively arranged in each area through the communication network. The servers 3 accept the commodity recognition information and return information of the commodities to the server 2. The server 2 integrates the commodity information transmitted from each server 3 and transmits it to the user terminal 1. Since only standard patterns of commodities are stored in the the database of the server 2, the increase in capacity can be prevented. Further, since the momentarily changing commodity information is stored in databases on the distributed shop server side, update management of the database is also facilitated. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a product search system and method, and a terminal device and a server device used therefor, and more particularly to a product search method for searching for detailed information on a product desired by a user, handling store information, and the like via a network.

  If a user wants to purchase a product or see the actual product that he / she wants to purchase, he / she needs to go to the store where the product is handled and confirm it or consult with the store clerk. is there. For this purpose, the user needs to know the name and location of the store in advance. When the user goes to the store, the product desired by the user may not be handled, or the actual product may not be viewed due to lack of stock.

  Recently, there is also a method for accessing the homepage of a store handling a product desired by a user via the Internet (Web) and confirming information on the product. In this case, accurate information cannot be obtained unless the search key such as the name of the desired product is known.

Therefore, Patent Literature 1 discloses a technique for obtaining information about a target product and information about a store, without using information such as the name of the product. That is, the image data of the product photographed using the camera function of the mobile phone terminal is transmitted to the server via the network. In this server, a product database, a handling store database, and a map database are provided, and feature amount information is extracted by performing image analysis on image data from a mobile phone terminal. Use this feature information as a search key to search the product database, obtain detailed product information and map information including the location of the store from the store database and map database, and provide it to the mobile phone terminal, that is, the user It is supposed to be.
JP 2003-122757 A

  In the technology disclosed in Patent Literature 1 described above, a product database, a handling store database, and a map database are provided in one server, and a product search request from a user's mobile phone terminal is received and these databases are searched. It has become. In other words, information related to products and stores is stored in the database of the server, and image analysis processing and pattern matching processing as well as product information extraction processing are all concentrated on this server alone. Yes.

  For this reason, there is a problem that the load on the server becomes extremely large, the time required for the processing becomes long, and the efficiency is low. In addition, the database also needs to store information on all products and stores that can be sufficiently satisfied with the search for the product desired by the user in the database, and the database capacity is extremely large. It becomes. Conversely, when the capacity is limited, there is a problem that the degree of satisfaction with the user becomes insufficient. Furthermore, since information on products and stores changes from moment to moment, it is necessary to update the database. However, since this information is stored in a centralized database in the server, There is also a problem that update management has to be extremely complicated.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a product search system and method, and a user terminal and server used therefor, which do not increase the capacity of a database for product search, eliminate the complexity of update management, and enable efficient search processing. Is to provide a device.

  A product search system according to the present invention is a product search system that performs a product search process based on a product search request, generates an image pattern of a search target product, and transmits the product search request as a product search request via a communication network. In response to receiving the product search request, the terminal device having a means for transmitting, and performing a pattern matching process between the image pattern and a standard pattern of products stored in the database in advance, From the information about the product stored in advance in the database in response to the first server device having means for transmitting the product recognition information for specifying via the communication network and receiving the product recognition information, the product A second server device having means for extracting product information related to the product specified by the recognition information and transmitting it to the first server device Wherein the door, to the terminal device from the first server, the extracted product information, characterized in that is to be sent.

  A product search method according to the present invention is a product search method for performing a product search process based on a product search request from a terminal device, wherein the terminal device generates an image pattern of a product to be searched and In the first server device, in response to receiving the product search request, the image pattern and a standard pattern of products stored in the database in advance are transmitted as a product search request via a communication network. A step of transmitting product recognition information for specifying the product of the matching result by performing pattern matching processing, and a second server device, in response to reception of the product recognition information, in the database The product information related to the product specified by the product recognition information is extracted from the information related to the product stored in advance, and the first Transmitting to the server device, the to the terminal device from the first server apparatus, characterized by comprising a step of transmitting the extracted product information.

  A terminal device according to the present invention is a terminal device in a product search system that performs a product search process based on a product search request, and generates an image pattern of a product to be searched and uses a communication network as the product search request. And means for making a refined search request for the search target product.

  An image processing server device according to the present invention is an image processing server device configured to perform a product search process based on a product search request including an image pattern of a search target product from a terminal device, and for receiving the product search request. In response, a means for generating a product recognition information for performing a pattern matching process between the image pattern and a standard pattern of a product stored in advance in a database and identifying a product as a result of the matching, and the product recognition And means for transmitting information to a store server device installed in various places via a communication network.

  The store server device according to the present invention performs a pattern matching process between the image pattern and the standard pattern of the product in response to the product search request including the image pattern of the search target product from the terminal device, and the product of the matching result A merchant server device that receives the product recognition information via a communication network and searches for product information from an image processing server device configured to generate product recognition information for specifying the product, In response to receiving the recognition information, including means for extracting product information related to the product specified by the product recognition information from information related to the product stored in advance in the database and transmitting the product information to the image processing server device. Features.

  A program according to the present invention is a program for causing a computer to execute an operation of a terminal device in a product search system configured to perform a product search process based on a product search request, and generates an image pattern of a product to be searched. And a process for transmitting the product search request via a communication network and a process for making a refined search request for the search target product.

  Another program according to the present invention is a program for causing a computer to execute an operation of an image processing server device that performs a product search process based on a product search request including an image pattern of a search target product from a terminal device. In response to receiving the product search request, product recognition information for performing a pattern matching process between the image pattern and a standard pattern of products stored in the database in advance and specifying the matching product And a process of transmitting the product recognition information to a store server device installed in various places via a communication network.

  Still another program according to the present invention performs a pattern matching process between the image pattern and the standard pattern of the product in response to a product search request including the image pattern of the search target product from the terminal device, and the product of the matching result The computer executes the operation of the store server device that receives the product recognition information via the communication network from the image processing server device configured to generate the product recognition information for specifying the product information and searches the product information. In response to receiving the product recognition information, the image processing is performed by extracting product information related to the product specified by the product recognition information from information related to the product stored in a database in advance. It includes a process of transmitting to a server device.

  The operation of the present invention will be described. An image pattern of a product desired by the user is transmitted from the user terminal to the image processing server device via the communication network. In this server device, the transmitted image pattern is compared with a prestored product standard pattern to search for similar / matching products by pattern matching. Merchandise recognition information for identifying the retrieved merchandise is transmitted via a communication network to a store server device distributed in various places. The dealer server device at each location receives the product recognition information, returns the product information related to the product to the image processing server device, and the image processing server device sends the corresponding product transmitted from each dealer server device. The information is integrated and transmitted to the user terminal. The user terminal can determine a desired product by looking at these product information. If there are a large number of corresponding products, a narrowing search is requested and the products can be narrowed down.

  The image processing server device functions as a mobile agent that is an entrance to the user terminal with respect to the product search service, and stores detailed information about products at each handling point as a database in the store server devices distributed in various places. In addition, since the image processing server device as a mobile agent can store only standard patterns related to products as a database, the capacity of the database of the image processing server device does not increase significantly, and sometimes Since the information on the products that change every moment is stored in the database on the distributed dealer server side, the update management of the data base becomes easy.

  According to the present invention, an image processing server device is provided as a mobile agent, and only the standard pattern of the product is stored in this database, and detailed information on the product is stored in a database on the distributed dealer server device side. Since the data is stored, an increase in the capacity of the data base of the image processing server apparatus is suppressed, and further, the product information can be easily updated and managed.

  In addition, users can easily and easily search for products that meet their wishes with respect to design images, and by using camera mobile phones that have become widespread, they can be used on the go including overseas. Also, there is an effect that it is possible to perform a similar search for a product that the user is interested in. In addition, there is an effect that a retailer may be able to acquire potential customers even for individuals who are difficult to place large-scale advertisements, small-scale retailers, select shops, and the like.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an overall system schematic diagram showing an embodiment of the present invention. As shown in FIG. 1, in the present embodiment, a user terminal 1 that is a portable mobile communication terminal such as a mobile phone or a PDA (Personal Digital Assistant), and an image processing server device (hereinafter simply referred to as a server) as a mobile agent. 2) and a store server 3 including servers 3a and 3b installed in an individual store and a mass store and an online shopping server 3c.

  Although not particularly shown, the user terminal 1, the image processing server 2, and the store server 3 are connected to each other via a known communication network (for example, the Internet). Although there are a plurality of user terminals 1, only one is shown in the figure for simplicity.

  FIG. 2 is a functional block diagram of the user terminal 1. The user terminal 1 has a function of transmitting / receiving information to / from the image processing server 2 via a communication network. For this purpose, the user terminal 1 can wirelessly communicate with a wireless base station (not shown). A communication unit 11 is provided. In addition, an imaging unit (camera) 12 that captures an image, a display unit 13 that displays various types of information such as images and texts, an information input operation unit 14 that includes numeric keys, scroll keys, and enter keys, and audio information Of the microphone / speaker for inputting / outputting, the control unit (CPU) 16 for controlling these units 11 to 15, telephone directory information, incoming / outgoing call information, transmission / reception mail information, captured image, terminal It has a memory 17 for storing various setting information and a program (application software) for controlling the operation of the product search system.

  FIG. 3 is a functional block diagram of the image processing server 2. The server 2 is an information processing device such as a workstation or server that is installed and operated and managed by a business provider that provides the product search service, and has a function as a mobile agent. The server 2 performs image recognition processing on the communication unit 21 that communicates with the user terminal 1 and the store server 3 via the communication network and the image transmitted from the user terminal 1 and stores the image in the database 23. The image recognition / pattern matching unit 22 that performs pattern matching processing (matching similarity search processing) with the standard pattern 232, information (user ID, etc.) 231 and image recognition / pattern matching related to the user who uses this product search service A database 23 storing a standard pattern 232 for the CPU, a control unit (CPU) 24 for controlling these units 21 to 23, and a memory functioning as a working memory for the CPU 24 while storing the operation procedure of the CPU 24 as a program 25.

  FIG. 4 is a functional block diagram of the store server 3 (3a to 3c). The server 3 is an information processing apparatus such as a workstation or server that is installed and managed by a business operator who designs, manufactures, or sells products. The server 3 receives predetermined product information from the database 33 based on the communication unit 31 that communicates with the image processing server 2 via the communication network and the product recognition information (product characteristics) transmitted from the image processing server 3. A product information extraction unit 32 to be extracted, a database 33 that stores information such as shape, pattern, color, price, appearance image, manufacturer name, and inventory (delivery date), which are various product information, and these units 31 and 32 are controlled. A control unit (CPU) 34 and a memory 35 that previously stores control operations of the CPU 34 as a program and functions as a working memory for the CPU 34 are provided.

  The operation of the embodiment of the present invention will be described below. Referring to FIG. 5, an operation flow diagram at the time of a product search request in the user terminal 1 is shown, and FIG. 6 is a diagram illustrating a transition example of the display screen of the display unit of the user terminal 1 according to the flow. When the user wants to receive this service, first, the display screen 101 in FIG. 6 is selected and displayed. “SISS” in the figure indicates a similar item search system, that is, a similar product search system.

  Either “pattern search” or “narrow search” on the display screen 101 can be selected. In general, when “pattern search” is generally selected and there are many corresponding products in the search result, “Selection search” is selected. Therefore, here, when “pattern search” is selected (step S1), the camera is activated (step S2). Thus, the user can shoot a product he / she wants. The display screen 102 in FIG. 6 is a screen for selecting and instructing “camera”. When “camera” is selected by the user, shooting is performed (step S3). Thereafter, the “search” shown in the display screen 103 of FIG. 6 is displayed. By selecting this, the search request is transmitted to the image processing server 2 (step S4).

  At this time, not only the image pattern photographed by the camera but also voice input (including sentence input by voice recognition) may be added, and the image pattern is a camera having a CCD or CMOS two-dimensional sensor. In addition, a scanner output by a one-dimensional line sensor can also be used. Furthermore, the image editing application software is used to edit (process) the image, or to add a feature point range designation marking to facilitate the search, For this purpose, the image pattern may be transmitted to the image processing server 2.

  FIG. 7 is a flowchart showing the operation of the image processing server 2. When a product search request including an image pattern is received from the user terminal 1 (step S11), the standard pattern 232 stored in the database 23 and the received image pattern are received. And image recognition / pattern matching processing is performed (step S12). This standard pattern 232 categorizes search target products into categories (for example, passenger cars, bags, clothes,...), And in each category, for example, a passenger car, becomes a standard for the vehicle type (for example, wagon, sedan, etc.). It is a pattern.

  If there is a standard pattern that matches the received image pattern by pattern matching processing, information about this standard pattern (information indicating the characteristics of the product, product recognition information indicating “wagon”, “sedan”, etc.) Then, it is transmitted to the distribution server 3 (3a-3c) that is distributed (step S13).

  FIG. 8 is a flowchart showing the operation of the store server 3. When product recognition information as a pattern matching result in the image processing server 2 is received (step S21), the database 33 is searched based on the product recognition information. (Step S22). For example, if the product recognition information is information indicating “sedan”, product information related to this sedan is extracted from the database 33 (step S23). In the case of a passenger car, the product information includes an entire image, a manufacturer name, a vehicle type, a price, catalog information, dealer information, inventory information (delivery date), and the like, and the extracted product information is transmitted to the image processing server 2. (Step S24).

  FIG. 9 is an operation flowchart of the image processing server 2 when the product information transmitted from the store server 3 is received. When the product information is received from the store server 3 (step S31), all the product information from the store server 3 is integrated and transmitted as a search result to the user terminal 1 that is the search request source (step S32).

  FIG. 10 is an operation flowchart of the user terminal 1 when the search result transmitted from the image processing server 2 is received. When the search result is received from the image processing server 2 (step S41), the search result is displayed on the display unit (step S42). An example of the display screen of the display unit in the user terminal 1 in this case is shown at 108 in FIG. In this example, the number of hits is 50, and when the user determines that narrowing is necessary, “narrowing search” on the display screen can be selected. This “squeezed search” is equivalent to the “squeezed search” on the display screen 101 in FIG. 6. When this “squeezed search” is selected (step S1 in FIG. 5), the display screen 104 in FIG. 6 is displayed. . Therefore, when the product category to be narrowed down is designated (as indicated by a1, “passenger car” is set: step S5 in FIG. 5), the designated category narrowing screen 105 is displayed (step S6 in FIG. 5).

  This screen 105 is for searching for passenger cars, and in addition to the shape and color, for each item indicating a partial feature such as an emblem or a headlight, an additional image a2 or a user's selection a3 is designated. It is possible. For example, if an “image” a2 of the shape is designated, an additional captured image for narrowing down can be captured. In order to make image recognition and pattern matching more accurate in the image processing server 2, a window frame is provided, It can also be configured such that an object is placed in the window frame and photography from the front is requested. When “selection” a3 is designated, a pattern 106 using icons as selection candidates registered in advance is displayed, and a desired one of them can be selected. Furthermore, by selecting “character input” a4, it is possible to input a search condition by characters.

  In the display screen 105 in FIG. 6, for an additional color image, correction such as white balance adjustment may be performed from the overall color balance of the captured color.

  In this way, when the refinement search request is transmitted to the image processing server S4 (step S4 in FIG. 5), the operations in FIGS. 7 to 10 are performed, and the refinement search result is displayed as shown in the display screen 109 in FIG. Will be displayed on the display unit of the user terminal 1. This narrowing search can also be performed by selecting “narrowing search” from the beginning on the display screen 101 in FIG. 6, and in this case, the process proceeds to the right side in step S <b> 1 of the flow in FIG. 5. If such a narrowing search process is repeatedly performed, further narrowing down becomes possible.

  When the user has a purchase request for the product searched as described above, if the user selects the desired product and indicates the purchase request, the user terminal 1 handles the purchase request for the product together with the user ID. It is sent to the store server. At this time, the image processing server 2 uses the user ID for authentication and collates it with the user information 231 in the database 23. To order products.

  Note that a reverse auction is also possible for the product searched by the above-described product search system. For example, in a state where a plurality of products are searched as a result of the narrowing search, the user inputs purchase conditions such as a desired purchase price and a desired purchase period, and registers them in the server 2 as a mobile agent. The server 2 presents this registration content to the dealer server side, and the dealer server side returns information that matches the purchase conditions of the user to the user via the server 2. In the user terminal 1, the purchase of the product that the user is most satisfied among the information from each store is determined.

  In the above-described embodiment, it is obvious that a mobile phone that can access a communication network such as the Internet can be used as the user terminal 1, and a PDA, a PC (personal computer), or the like can be used. In addition, each operation of the user terminal and the server can be stored in advance in a recording medium as a program and read and executed by a CPU (computer).

1 is a schematic system configuration diagram of an embodiment of the present invention. It is a functional block diagram of a user terminal. It is a functional block diagram of an image processing server. It is a functional block diagram of a store server. It is an operation | movement flowchart at the time of the search request | requirement in a user terminal. It is a figure which shows the transition of the display screen at the time of the search request | requirement in a user terminal. It is an operation | movement flowchart at the time of the search in an image processing server. It is an operation | movement flowchart at the time of the search in a store server. It is an operation | movement flowchart in the case of returning the search result in an image processing server to a user terminal. It is an operation | movement flowchart at the time of the search result reception in a user terminal. It is a figure which shows the example of the display screen at the time of the refinement | retrieval search in a user terminal.

Explanation of symbols

1 User terminal
2 Image processing server
3 dealer server
11 Wireless communication unit
12 Imaging unit
13 Display section
14 Operation unit
15 Transmission / Reception Unit 16, 24, 34 Control Unit (CPU)
17, 25, 35 memory
21, 31 Communication Department
22 Image recognition / pattern matching section
23,33 database
32 Product information extraction unit
231 User information
232 standard pattern

Claims (27)

A product search system that performs a product search process based on a product search request,
A terminal device having means for generating an image pattern of a product to be searched and transmitting it as a product search request via a communication network;
In response to receiving the product search request, product recognition information is communicated to perform a pattern matching process between the image pattern and a standard pattern of products stored in the database in advance and identify the matching product. A first server device having means for transmitting via a network;
Means for extracting product information relating to a product specified by the product recognition information from information relating to a product stored in advance in a database in response to receiving the product recognition information, and transmitting the product information to the first server device; A second server device comprising:
And the extracted product information is transmitted from the first server device to the terminal device.   2. The commodity search system according to claim 1, wherein the second server device is a server device installed at a store in each place.   The product search system according to claim 1, wherein the terminal device enables input by voice or text as the product search request.   The product search system according to any one of claims 1 to 3, wherein the terminal device further includes a narrow search request means for making a narrow search request for the search target product.   5. The product search system according to claim 4, wherein the narrowing-down search request means enables a partial image of the product to be searched to be input.   6. The commodity search system according to claim 5, wherein the narrowing search request means further enables input by voice or character input.   7. The product search system according to claim 5 or 6, wherein the narrowing search request means further presents a selection candidate predetermined for the product and enables designation input among the selection candidates.   The commodity search system according to any one of claims 5 to 7, wherein the narrow-down search requesting means repeatedly performs the narrow-down search to narrow down sequentially. A product search method for performing a product search process based on a product search request from a terminal device,
In the terminal device, generating an image pattern of a product to be searched and transmitting it as a product search request via a communication network;
In response to receiving the product search request, the first server device performs a pattern matching process between the image pattern and a standard pattern of products stored in advance in the database, and specifies a product as a result of the matching. Transmitting product recognition information for communication via a communication network;
In the second server device, in response to receiving the product recognition information, product information related to the product specified by the product recognition information is extracted from information related to the product stored in the database in advance. Transmitting to the server device;
And a step of transmitting the extracted product information from the first server device to the terminal device.   The method according to claim 9, wherein the second server device is a server device installed at a store in each place.   The product search method according to claim 9 or 10, wherein the terminal device enables input by voice or text as the product search request.   The product search method according to any one of claims 9 to 11, further comprising a narrow search request step of making a narrow search request for the search target product in the terminal device.   13. The product search method according to claim 12, wherein the narrowing-down search request step allows a partial image of the product to be searched to be input.   14. The product search method according to claim 13, wherein the narrowing search request step further enables input by voice or character input.   15. The product search method according to claim 13 or 14, wherein the narrowing search request step further presents a predetermined selection candidate for the product and enables specification input among the selection candidates.   The commodity search method according to any one of claims 13 to 15, wherein in the narrowing search request step, the narrowing search request is repeatedly performed to narrow down sequentially. A terminal device in a product search system that performs a product search process based on a product search request,
Means for generating an image pattern of a product to be searched and transmitting it as a product search request via a communication network;
A refinement search request means for making a refinement search request for the search target product;
The terminal device characterized by including.   The terminal device according to claim 17, wherein the narrowing-down search request unit enables input of a partial image of the search target product.   19. The terminal device according to claim 18, wherein the narrowing-down search request unit further enables input by voice or character input.   The terminal device according to claim 18 or 19, wherein the narrow-down search request means further presents a predetermined selection candidate for the product and enables designation input among the selection candidates.   The terminal device according to any one of claims 18 to 20, wherein the narrowing-down search requesting unit sequentially narrows down the narrowing-down search. An image processing server device configured to perform a product search process based on a product search request including an image pattern of a search target product from a terminal device,
In response to receiving the product search request, pattern matching processing is performed between the image pattern and a standard pattern of products stored in advance in a database, and product recognition information for specifying the product of the matching result is generated. Means to
Means for transmitting the product recognition information to a store server device installed in various places via a communication network;
An image processing server device comprising: In response to a product search request including an image pattern of a search target product from a terminal device, product recognition information for performing a pattern matching process between the image pattern and a standard pattern of the product and specifying a product as a result of the matching is performed. A store server device that receives the product recognition information via a communication network and searches for product information from an image processing server device configured to generate;
Means for extracting product information related to the product specified by the product recognition information from information related to the product stored in advance in a database in response to receiving the product recognition information, and transmitting the product information to the image processing server device A store server device characterized by that.   The dealer server device according to claim 23, which is installed in a dealer in each place. A program for causing a computer to execute an operation of a terminal device in a product search system configured to perform a product search process based on a product search request,
A process of generating an image pattern of a product to be searched and transmitting it as a product search request via a communication network;
A process for making a narrow search request for the search target product;
The program characterized by including. A program for causing a computer to execute an operation of an image processing server device configured to perform a product search process based on a product search request including an image pattern of a search target product from a terminal device,
In response to receiving the product search request, processing for generating product recognition information for specifying a product of the matching result by performing a pattern matching process between the image pattern and a standard pattern of the product;
A process of transmitting the product recognition information to a store server device installed in various places via a communication network;
The program characterized by including. In response to a product search request including the image pattern of the search target product from the terminal device, pattern matching processing is performed between the image pattern and the standard pattern of the product stored in advance in the database, and the matching result product is obtained. From the image processing server device configured to generate product recognition information for identification, the computer executes an operation of the store server device that receives the product recognition information via a communication network and searches for product information. A program for
In response to receiving the product recognition information, processing for extracting product information related to the product specified by the product recognition information from information related to the product stored in advance in a database and transmitting the product information to the image processing server device. A program characterized by that.

Images