JP2013186548A - Information processing device, store system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable more efficient selection of a product corresponding to an imaged object.SOLUTION: An information processing device of an embodiment comprises: image import means for importing images taken by imaging means; similarity calculation means for calculating similarity which indicates how similar a feature quantity of a product that is previously stored and a feature quantity of an object that is imaged by the imaging means are, for each product registered in product master data; display control means for causing display means to display a list of products registered in the product master data on the basis of the product master data in a way that differentiates the display mode of products having higher similarity from that of other products in the list; and selection receiving means for receiving selection of a product in the list.

Description

  Embodiments described herein relate generally to an information processing apparatus, a store system, and a program.

  Conventionally, for general object recognition that recognizes (detects) the type or the like of an article by extracting the feature quantity of the object from image data obtained by imaging the article and comparing it with data (feature quantity) for collation prepared in advance. Such technology exists. In addition, a system has been proposed in which the technology related to general object recognition is used for discrimination of products and the like.

  By the way, in the technique related to the general object recognition described above, a plurality of products that are candidates for the target may be recognized. In such a case, the operator selects a candidate product, but if the candidate product is displayed carelessly by a pop-up or the like, the selection of the product may be hindered. For example, shop assistants who are used to operations are more familiar with the product list in the product master data, so it is more efficient to select from a familiar product list than to display candidate products in a pop-up. It was the target.

  In order to solve the above-described problems, an information processing apparatus according to an embodiment includes an image capturing unit that captures an image captured by an imaging unit, and a feature of the product that is stored in advance for each product registered in the product master data. Based on the product master data, similarity calculation means for calculating the similarity indicating how much the amount and the feature quantity of the object imaged by the imaging means are similar to the product master data A display control means for displaying a list of each registered product on a display means, and displaying a display mode of a product with a high similarity in the list different from other products, and in the list Selection accepting means for accepting selection of the product.

  In addition, the store system of the embodiment includes an imaging unit, an image capturing unit that captures an image captured by the imaging unit, and a feature amount of the product that is stored in advance for each product registered in the product master data. Registered in the product master data on the basis of the similarity calculation means for calculating the similarity indicating how similar the feature quantity of the object imaged by the imaging means is, and the product master data A display list for each product is displayed on the display means, a display control means for displaying a product with a high similarity in the list in a display mode different from other products, and a list of products in the list Selection receiving means for receiving selection; and sales registration processing means for performing sales registration processing of the selected product.

  In addition, the program according to the embodiment includes a computer, an image capturing unit that captures an image captured by the imaging unit, a feature amount of the product stored in advance for each product registered in the product master data, and the imaging unit. Based on the product master data, similarity calculation means for calculating the degree of similarity indicating the degree of similarity between the feature quantities of the objects imaged in step 1 and each product registered in the product master data. Display list on display means, display control means for displaying products with high similarity in list in display mode different from other products, and accept selection of products in the list It functions as a selection receiving means.

FIG. 1 is a perspective view illustrating an example of a checkout system (store system) according to the embodiment. FIG. 2 is a block diagram illustrating a hardware configuration of the POS terminal and the commodity reading apparatus. FIG. 3 is a conceptual diagram illustrating the data structure of a PLU file. FIG. 4 is a block diagram showing a functional configuration realized by the CPU sequentially executing the program. FIG. 5 is a flowchart showing an example of the operation of the checkout system according to the present embodiment. FIG. 6 is a conceptual diagram illustrating an example of a reading area in the reading window. FIG. 7 is a flowchart illustrating an example of the operation of the product selection process. FIG. 8 is a conceptual diagram illustrating an example of a product list for each classification. FIG. 9 is a conceptual diagram showing an example of the selection screen. FIG. 10 is a conceptual diagram illustrating an example of a selection screen. FIG. 11 is a conceptual diagram illustrating an example of the selection screen.

  Hereinafter, an information processing apparatus, a store system, and a program according to the present embodiment will be described with reference to the drawings, taking a checkout system as an example. The store system is a checkout system (POS system) provided with a POS terminal that performs registration and settlement of commodities related to one transaction. This embodiment is an application example to a checkout system introduced in a store such as a supermarket.

  FIG. 1 is a perspective view illustrating an example of a checkout system 1 (store system) according to the embodiment. As shown in FIG. 1, the checkout system 1 includes a POS terminal 11 that performs registration and settlement of commodities related to one transaction. The POS terminal 11 is placed on the upper surface of the drawer 21 on the checkout table 51. The drawer 21 is controlled by the POS terminal 11 for the opening operation. On the upper surface of the POS terminal 11, a keyboard 22 that is pressed down by an operator (a store clerk) is arranged. A display device 23 that displays information toward the operator is provided behind the keyboard 22 when viewed from the operator who operates the keyboard 22. The display device 23 displays information on the display surface 23a. A touch panel 26 is laminated on the display surface 23a. A customer display device 24 is erected so as to be rotatable further behind the display device 23. The customer display device 24 displays information on the display surface 24a. The customer display device 24 shown in FIG. 1 has the display surface 24a facing the front side in FIG. 1, but the customer display device 24 is rotated so that the display surface 24a faces the back side in FIG. Thus, the customer display device 24 displays information to the customer.

  A horizontally long table-shaped counter table 151 is arranged so as to form an L shape with the checkout table 51 on which the POS terminal 11 is placed. A cargo receiving surface 152 is formed on the upper surface of the counter table 151. A shopping basket 153 for storing the product G is placed on the cargo receiving surface 152. The shopping cart 153 may be divided into a first shopping cart 153a brought in by a customer and a second shopping cart 153b placed at a position sandwiching the commodity reading device 101 from the first shopping cart 153a. it can. The shopping basket 153 is not limited to a so-called basket shape, and may be a tray or the like. The shopping basket 153 (second shopping basket 153b) is not limited to a so-called basket shape, and may be a box shape, a bag shape, or the like.

  On the receiving surface 152 of the counter table 151, the commodity reading apparatus 101 connected to the POS terminal 11 so as to be able to transmit and receive data is installed. The commodity reading apparatus 101 includes a thin rectangular housing 102. A reading window 103 is disposed in front of the housing 102. A display / operation unit 104 is attached to the upper portion of the housing 102. The display / operation unit 104 is provided with a display device 106 on which a touch panel 105 is laminated. A keyboard 107 is disposed on the right side of the display device 106. A card reading groove 108 of a card reader (not shown) is provided on the right side of the keyboard 107. A customer display device 109 for providing information to the customer is installed on the back left side of the display / operation unit 104 as viewed from the operator.

  Such a product reading apparatus 101 includes a product reading unit 110 (see FIG. 2). The merchandise reading unit 110 has an imaging unit 164 (see FIG. 2) disposed behind the reading window 103.

  In the first shopping basket 153a brought in by the customer, a product G related to one transaction is stored. The commodity G in the first shopping basket 153a is moved to the second shopping basket 153b by an operator who operates the commodity reading apparatus 101. In this movement process, the product G is directed to the reading window 103 of the product reading apparatus 101. At this time, the imaging unit 164 (see FIG. 2) arranged in the reading window 103 images the product G.

  The merchandise reading apparatus 101 displays a screen for designating which merchandise G included in the image captured by the imaging unit 164 corresponds to which merchandise registered in a PLU file F1 (see FIG. 3) described later. Displayed on the operation unit 104 and notifies the POS terminal 11 of the product ID of the specified product. In the POS terminal 11, based on the product ID notified from the product reading device 101, information related to sales registration such as the product classification, product name, and unit price of the product corresponding to the product ID is stored in a sales master file (not shown) or the like. And record sales.

  FIG. 2 is a block diagram illustrating a hardware configuration of the POS terminal 11 and the commodity reading apparatus 101. The POS terminal 11 includes a microcomputer 60 as an information processing unit that executes information processing. The microcomputer 60 is configured by connecting a ROM (Read Only Memory) 62 and a RAM (Random Access Memory) 63 to a CPU (Central Processing Unit) 61 that executes various arithmetic processes and controls each unit.

  The drawer 61, the keyboard 22, the display device 23, the touch panel 26, and the customer display device 24 are all connected to the CPU 61 of the POS terminal 11 through various input / output circuits (all not shown). . These are controlled by the CPU 61.

  The keyboard 22 includes a numeric key 22d, a temporary fastening key 22e, and a fastening key 22f on which numbers such as “1”, “2”, “3”,...

  An HDD 64 (Hard Disk Drive) is connected to the CPU 61 of the POS terminal 11. The HDD 64 stores programs and various files. All or a part of the programs and various files stored in the HDD 64 are copied to the RAM 63 and sequentially executed by the CPU 61 when the POS terminal 11 is activated. An example of a program stored in the HDD 64 is a product sales data processing program PR. An example of a file stored in the HDD 64 is a PLU file F1 distributed and stored from the store computer SC.

  The PLU file F1 includes product master data such as a product ID for identifying a product, a classification to which the product belongs (product classification), a sales price (unit price), and an image of the product for each of the products displayed and sold in the store. It is a recorded file.

  FIG. 3 is a conceptual diagram illustrating the data configuration of the PLU file F1. As shown in FIG. 3, the PLU file F1 includes, for each product G, a product ID uniquely assigned to the product, information on the product such as a product classification to which the product G belongs, a product name, a unit price, and a product obtained by capturing the product G. This is a file that stores an image and a threshold value of “similarity: 0.XX” (“0.50” in the illustrated example) as master data of the product G. Although the details will be described later, the threshold value of “similarity: 0.XX” indicates that the product G is a fruit or a fresh product, and the product stored in the PLU file F1 in advance when the freshness is deteriorated or discolored. By comparing with the product image, it is possible to determine that the product is different from the normal state. The PLU file F1 can be read (referenced) by the product reading apparatus 101 via a connection interface 65 described later.

  Note that the data structure of the PLU file F1 is not limited to the example of FIG. 3, and for example, feature quantities (color shades, surface irregularities, etc.) described later for each product that can be read from the product image are stored in association with each other. It is good also as a form to do.

  Returning to FIG. 2, a communication interface 25 for executing data communication with the store computer SC is connected to the CPU 61 of the POS terminal 11 via an input / output circuit (not shown). The store computer SC is installed in a store backyard or the like. A PLU file F1 distributed to the POS terminal 11 is stored in the HDD (not shown) of the store computer SC.

  Further, the CPU 61 of the POS terminal 11 is connected to a connection interface 65 that enables data transmission / reception with the commodity reading apparatus 101. The product reading apparatus 101 is connected to the connection interface 65. The CPU 61 of the POS terminal 11 is connected to a printer 66 that prints on receipts. The POS terminal 11 prints the transaction content of one transaction on the receipt under the control of the CPU 61.

  The commodity reading apparatus 101 also includes a microcomputer 160. The microcomputer 160 is configured by connecting a ROM 162 and a RAM 163 to a CPU 161 via a bus. The ROM 162 stores a program executed by the CPU 161. The CPU 161 is connected to an imaging unit 164 and an audio output unit 165 via various input / output circuits (none of which are shown). The operations of the imaging unit 164 and the audio output unit 165 are controlled by the CPU 161. The display / operation unit 104 is connected to the product reading unit 110 and the POS terminal 11 via the connection interface 176. The operation of the display / operation unit 104 is controlled by the CPU 161 of the product reading unit 110 and the CPU 61 of the POS terminal 11.

  The imaging unit 164 is a color CCD image sensor, a color COMS image sensor, or the like, and is an imaging unit that performs imaging from the reading window 103 under the control of the CPU 161. For example, the imaging unit 164 captures a 30 fps moving image. Frame images (captured images) sequentially captured at a predetermined frame rate by the imaging unit 164 are stored in the RAM 163.

  The audio output unit 165 is an audio circuit and a speaker for generating a preset warning sound or the like. The sound output unit 165 performs notification by sound such as a warning sound under the control of the CPU 161.

  Furthermore, a connection interface 175 that connects to the connection interface 65 of the POS terminal 11 and enables data transmission / reception with the POS terminal 11 is connected to the CPU 161. In addition, the CPU 161 transmits and receives data to and from the display / operation unit 104 via the connection interface 175.

  Next, functional units realized by the CPU 161 and the CPU 61 sequentially executing programs will be described with reference to FIG.

  FIG. 4 is a block diagram illustrating a functional configuration realized by the CPU 161 and the CPU 61 sequentially executing programs. As illustrated in FIG. 4, the CPU 161 of the product reading apparatus 101 sequentially executes the program, thereby causing the image capturing unit 1611, the product detection unit 1612, the similarity calculation unit 1613, the product selection unit 1614, and the registered product notification unit 1615. As a function. Similarly, the CPU 61 of the POS terminal 11 has a function as the sales registration unit 611.

  The image capturing unit 1611 outputs an imaging on signal to the imaging unit 164 and causes the imaging unit 164 to start an imaging operation. The image capturing unit 1611 sequentially captures frame images captured by the image capturing unit 164 and stored in the RAM 163 after the start of the image capturing operation. The frame image capture by the image capture unit 1611 is performed in the order stored in the RAM 163.

  The product detection unit 1612 detects all or part of the product G included in the frame image captured by the image capture unit 1611 using a pattern matching technique or the like. Specifically, a contour line or the like is extracted from an image obtained by binarizing the captured frame image. Next, the contour line extracted from the latest frame image is compared with the contour line extracted from the current frame image, and the part that has changed, that is, the product directed to the reading window 103 for sales registration G reflection is detected.

  As another method for detecting the product G, the presence or absence of a skin color region is detected from the captured frame image. Next, when the skin color area is detected, that is, when the reflection of the clerk's hand is detected, the contour line is detected as described above, so that the clerk's hand is assumed to be grasped by the hand. Attempt contour extraction. At this time, when the contour indicating the shape of the hand and the other contour are detected, the reflection of the product is detected because the clerk's hand is holding the product.

  The similarity calculation unit 1613 reads the state of the surface such as the hue of the product G and the unevenness of the surface from the whole or a part of the image of the product G captured by the imaging unit 164 of the product reading apparatus 101 as a feature amount. Note that the similarity calculation unit 1613 does not consider the outline or size of the product G in order to shorten the processing time.

  In addition, the similarity calculation unit 1613 reads the surface state such as the color of the registered product and the surface unevenness from the product image of each product registered in the PLU file F1 (hereinafter referred to as registered product) as a feature amount. The similarity between the product G and the product registered in the PLU file F1 is calculated by comparing with the feature amount of the product G. Here, the similarity is the degree of similarity of all or part of the product G when the product image of each product stored in the PLU file F1 is 100% = “similarity: 1.0”. It shows whether you are doing. As described above, the degree of similarity is calculated according to the state of the surface such as the hue and the surface roughness. In addition, for example, the weight may be changed according to the color tone and the surface roughness.

  In addition, the similarity calculation unit 1613 determines whether the similarity calculated for each registered product exceeds a threshold (“similarity: 0.XX”) predetermined for the product, and the similarity is determined. Registered products exceeding this threshold are recognized (determined) as candidates for the product G (hereinafter referred to as product candidates). In addition, when the feature-value of each product image is matched and stored in PLU file F1, it is good also as a form compared using the feature-value stored in PLU file F1.

Recognizing an object contained in an image in this way is called generic object recognition. Regarding such general object recognition, various recognition techniques are described in the following documents.
Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

Further, techniques for performing general object recognition by dividing an image into regions for each object are described in the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [August 10, 2010 search], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>

  Note that the method of calculating the similarity between the captured image of the product G and the product image of the registered product registered in the PLU file F1 is not particularly limited. For example, the similarity between the captured image of the product G and each registered product registered in the PLU file F1 may be calculated as an absolute evaluation or may be calculated as a relative evaluation.

  When the former calculation method is used, if the captured image of the product G and each registered product registered in the PLU file F1 are compared on a one-to-one basis, the similarity derived as a result of this comparison is used as it is. Good. When the latter calculation method is used, for example, if five registered products (products GA, GB, GC, GD, GE) are registered in the PLU file F1, the captured product G is The similarity is 0.6, the similarity is 0.1 for the product GB, the similarity is 0.1 for the product GC, the similarity is 0.1 for the product GD, and the product GE On the other hand, the similarity is calculated to be 0.1 (100%), for example, the similarity is 0.1, and the total similarity with each registered product.

  The product selection unit 1614 functions as a display control unit that displays a list of products registered in the PLU file F1 on the display device 106 based on the PLU file F1. In addition, the product selection unit 1614 functions as a selection reception unit that receives a selection of a product in the list by receiving a touch operation on the touch panel 105 from the operator. The product selection unit 1614 outputs the information (for example, product ID, product name, etc.) indicating the product to the registered product notification unit 1615 as a product for which sales are registered as the product for which the selection has been accepted. In addition, the product selection unit 1614 displays, in a list displayed on the display device 106, products having a high similarity calculated by the similarity calculation unit 1613 in a display mode different from other products. The details of the product selection process related to the display of the list and the selection of products performed by the product selection unit 1614 will be described later.

  In this embodiment, when the product candidate is a single registered product, the registered product is the product G, and the process of selecting the product from the list is omitted. The present invention is not limited to this, and a single product candidate may be displayed on the display device 106 in a display mode different from that for other products so as to request confirmation from the operator. In the present embodiment, the product selection unit 1614 displays a list of product names on the display device 106, but may be a list of other product information, for example, character information such as product names and product prices. May be displayed, or the character information and the product image may be displayed together.

  The registered product notification unit 1615 notifies the POS terminal 11 of the product ID corresponding to the registered product instructed by the product selection unit 1614 together with the sales number separately input via the touch panel 105 or the keyboard 107. The notification of the product ID may be a form in which the registered product notification unit 1615 directly notifies the product ID read from the PLU file F1, or the file name or the product name of the product image that can specify the product ID. It is good also as a form which notifies, It is good also as a form which notifies the storage location (storage address in PLU file F1) of the goods ID to the POS terminal 11.

  The sales registration unit 611 registers the sales of the corresponding product based on the product ID and the number of sales notified by the registered product notification unit 1615. Specifically, the sales registration unit 611 refers to the PLU file F1, and records the notified product ID and the product classification, product name, unit price, etc. corresponding to the product ID together with the number of sales in a sales master file or the like. To register sales.

  Next, the operation of the checkout system 1 will be described in detail. FIG. 5 is a flowchart showing an example of the operation of the checkout system 1 according to the present embodiment.

  First, the operation on the commodity reading apparatus 101 side will be described. As illustrated in FIG. 5, when processing is started in response to the start of product registration by the POS terminal 11, the image capturing unit 1611 outputs an imaging on signal to the imaging unit 164 and performs imaging by the imaging unit 164. Start (S11).

  Next, the image capturing unit 1611 captures a frame image (image data) captured by the image capturing unit 164 and stored in the RAM 163 (S12). Next, the product detection unit 1612 detects all or part of the product G from the frame image captured by the image capture unit 1611 (S13). Next, the similarity calculation unit 1613 calculates the similarity by reading the feature amount of the product G from all or part of the image of the product G and comparing it with the feature amount of each product image registered in the PLU file F1. (S14).

  Next, the similarity calculation unit 1613 determines whether or not the similarity calculated for each of the registered products exceeds a threshold (“similarity: 0.XX”) predetermined for the product, and the similarity is Registered products exceeding the threshold are extracted as product candidates for the product G (S15).

  FIG. 6 is a conceptual diagram illustrating an example of the reading region R in the reading window 103. Specifically, FIG. 6 is a conceptual diagram illustrating the reading region R when the product G is read. As shown in FIG. 6, when the product G is reflected in the reading area R in the process of moving the product G by the operator's (store clerk) hand H, the product G is displayed in S13 from the frame image obtained by capturing the reading area R. Are detected in whole or in part. By detecting all or part of the product G, the product G is recognized in S14.

  Next, the product selection unit 1614 determines whether or not there are a plurality of product candidates (registered products) recognized in S15 (S16). Here, when it is determined that the number of registered products set as product candidates is one (S16: No), the product selection unit 1614 determines that this registered product corresponds to the product G. Then, the product selection unit 1614 outputs information indicating the registered product to the registered product notification unit 1615 together with the number of sales separately input, and proceeds to S18.

  On the other hand, when it is determined that the number of product candidates (registered products) is plural (S16: Yes), the product selection unit 1614 proceeds to the product selection process of S17. Here, the operation of the product selection process in S17 will be described.

  FIG. 7 is a flowchart illustrating an example of the operation of the product selection process. As shown in FIG. 7, when the product selection process is started, the product selection unit 1614 stores the product data (product ID, product classification, product name, unit price, product image, etc.) that is the product candidate in the PLU file F1. (S171). Next, the product selection unit 1614 determines the category of the products to be displayed in a list based on the category to which the product that is the product candidate belongs (S172).

  Specifically, the product selection unit 1614 sets the category to which the product that is the product candidate belongs as the category of the product to be displayed in a list. In addition, when the classification of each product that is a product candidate is different from each other, a category with a large number of products that are product candidates may be used as a category of products to be displayed in a list.

  Next, the product selection unit 1614 displays a product selection screen as a product candidate on the display device 106 and accepts a touch operation on the touch panel 105 from the operator, so that the product in the list displayed on the selection screen is displayed. The selection is accepted (S173). Specifically, the product selection unit 1614 is determined based on the PLU file F1 from the list created for each product category registered in the PLU file F1, as a product category to be displayed as a list. Display the selected list on the selection screen. Then, the product selection unit 1614 displays a product having a high similarity calculated by the similarity calculation unit 1613 in the list displayed on the selection screen, that is, a product that is a product candidate, separately from other products. Display in a manner.

  FIG. 8 is a conceptual diagram illustrating an example of the product list T1, T2, and T3 for each classification. As shown in FIG. 8, the product list T1, T2, and T3 are stored in the PLU file F1 for each product category (for example, “vegetable”, “meat”, “fruit”, etc.) registered in the PLU file F1. Are registered vertically and horizontally under predetermined conditions such as product ID order, product name alphabetical order, and unit price order. The product list tables T1, T2, and T3 may be created when each product registered in the PLU file F1 is updated, and may be stored in advance in the HDD 64, or may be stored when the selection screen is displayed. F1 may be read and created.

  FIG. 9 is a conceptual diagram showing an example of the selection screen G10. As shown in FIG. 9, in S173, the product list T1 determined as the category of products to be displayed in a list is displayed on the selection screen G10. Here, among the products displayed in the product list T1, the products T11, T12, and T13 that are product candidates are displayed differently from the other products due to the shaded display.

  As described above, the products T11, T12, and T13 can be easily discriminated by changing the display mode of the products T11, T12, and T13, which are the product candidates, to a display mode different from other products. Further, as long as there is no change in the PLU file F1, there is no change in the order of products in the product list T1, T2, T3. Therefore, the familiar sales clerk is familiar with the order of products in the product list T1, T2, T3. It will be. Therefore, a trained salesclerk can intuitively identify the products T11, T12, and T13 that are product candidates from the positions of the products T11, T12, and T13 having different display modes, and can efficiently select the products. Can do.

  In addition, on the selection screen G10, a selection button B for switching the product list T1, T2, T3 for each product category is displayed. In the selection button B, the selection buttons B11 and B12 corresponding to the category to which the product as the product candidate belongs are shaded and displayed as a display mode different from the selection buttons of other categories. As described above, the display mode of the selection button B11 and the selection button B12 corresponding to the category to which the product that is the product candidate belongs is set to a display mode that is different from the other categories, so that The list is easy to distinguish.

  Note that the product selection unit 1614 may change the display modes of the products T11, 12, 13, and the selection buttons B11, B12 described above. For example, the product selection unit 1614 may change the density of the shaded display in descending order of the similarity calculated by the similarity calculation unit 1613 for the products T11, T12, and T13. In this way, for the products T11, T12, and T13 that are the product candidates, the products with high similarity calculated by the similarity calculation unit 1613 are obtained by changing the shaded display density in descending order of similarity. It can be easily understood. Similarly, the product selection unit 1614 may change the density of the shaded display in the descending order of the number of products that are product candidates for the selection buttons B11 and B12.

  FIG. 10 is a conceptual diagram showing an example of the selection screen G20. As illustrated in FIG. 10, the product selection unit 1614 may change the display mode by surrounding line display. Specifically, the line type of the lines displayed around the frames of the products T11, T12, T13 and the selection buttons B11, B12 may be different from those of other products and selection buttons (thick lines in the illustrated example). Further, the line types in the products T11, 12, 13 and the selection buttons B11, B12 may be made thicker and different from each other in descending order of similarity and in descending order of the number of products that are product candidates. Further, the color of the line displayed around the frames of the products T11, T12, T13 and the selection buttons B11, B12 may be different from those of other products and selection buttons. In general object recognition, the degree of similarity between the registered product and the product G changes depending on the way and angle of the product G to be recognized, so that the recognition accuracy may not be sufficient. Therefore, as described above, presenting a plurality of product candidates and confirming the operator is effective when performing accurate product registration using general object recognition.

  FIG. 11 is a conceptual diagram showing an example of the selection screen G30. As illustrated in FIG. 11, the product selection unit 1614 may add an icon image to change the display mode. Specifically, the product selection unit 1614 displays the icon images G31, G32, and G33 of the products T11, T12, and T13 at positions corresponding to the products T11, T12, and T13. The icon images G31, G32, and G33 may be created by reading product images corresponding to the products T11, T12, and T13 from the PLU file F1. In addition, the product selection unit 1614 displays icon images G41 and G42 corresponding to the classification of the selection buttons B11 and B12 at positions corresponding to the selection buttons B11 and B12. The icon images G41 and G42 may be created in advance based on a product image of a symbolic product from among products belonging to the category, or a product image of a product belonging to the category may be randomly selected from the PLU file F1. It may be created by reading out.

  Subsequent to S173, the product selection unit 1614 determines whether or not the classification is changed by the selection operation of the selection button B (S174). When there is a change in the classification due to the selection operation of the selection button B (S174: YES), the product selection unit 1614 returns the process to S172, and displays the categories selected by the selection operation of the selection button B as a list of products for display. The selection screen will be displayed.

  When the classification is not changed by the selection operation of the selection button B (S174: NO), the product selection unit 1614 determines whether the product displayed in the list is selected by the touch operation of the touch panel 105 ( S175). If the product has not been selected (S175: NO), the product selection unit 1614 returns the process to S174. When the product is selected (S175: YES), the product selection unit 1614 sets the selected product as a product whose sales are registered (S176), and ends the process.

  Returning to FIG. 5, the registered merchandise notification unit 1615 stores the merchandise ID corresponding to the merchandise to be registered for sale instructed from the merchandise selection unit 1614 together with the sales quantity separately input via the touch panel 105 or the keyboard 107 to the POS terminal 11. Notification is made (S18). Here, the sales quantity is input via the touch panel 105 or the display / operation unit 104. However, the input method is not particularly limited. For example, the number of touches on the product image displayed on the display device 106 is sold. It is good also as a number.

  Next, the CPU 161 determines the presence / absence of business end from the POS terminal 11 by a product registration end notification or the like (S19). When the business is continued (S19: No), the CPU 161 returns the process to S12 and continues the process. When the business is to be ended (S19: Yes), the image capturing unit 1611 outputs an imaging off signal to the imaging unit 164, ends imaging by the imaging unit 164 (S20), and ends the process.

  Next, the operation on the POS terminal 11 side will be described. As shown in FIG. 5, when the process is started in response to the start of product registration in response to an operation instruction on the keyboard 22, the CPU 61 notifies the product ID of the product G and the number of items sold thereof notified from the product reading device 101. Is received (S31).

  Next, the sales registration unit 611 reads the product type, unit price, and the like from the PLU file F1 based on the product ID and the sales quantity received in S31, and registers the sales of the product G read by the product reading device 101 (S32). ).

  Subsequently, the CPU 61 determines whether or not there is an end of business due to the end of sales registration or the like by an operation instruction of the keyboard 22 (S33). When the business is continued (S33: No), the CPU 61 returns to S31 again and continues the process. When the business is to be ended (S33: Yes), the CPU 61 ends the process.

  As described above, in the commodity reading apparatus 101 as the information processing apparatus, the image captured by the image capturing unit 164 is captured, and for each product registered in the PLU file F1, the pre-stored product feature amount and the image capturing unit 164 are stored. The similarity calculation unit 1613 calculates a similarity indicating how similar the feature quantity of the object imaged in step 1 is. The product selection unit 1614 of the product reading apparatus 101 displays a list of products registered in the PLU file F1 on the display device 106 based on the PLU file F1, and stores products with high similarity in the list. The display mode is displayed differently from other products. In addition, the product selection unit 1614 accepts selection of a product in the list by the touch panel 105. Therefore, a skilled clerk (operator) can intuitively identify a product having a high similarity from the position of a product with a different display mode in the familiar list, and the product corresponding to the imaged object can be identified. Selection can be performed more efficiently.

  Note that the program executed by each device of the present embodiment is provided by being incorporated in advance in a ROM or the like. A program executed in each apparatus of the present embodiment is a file in an installable format or an executable format, and is read by a computer such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). You may comprise so that it may record on a possible recording medium and provide.

  Furthermore, the program executed by each device of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed in each device of the present embodiment may be configured to be provided or distributed via a network such as the Internet. Besides, the master data of the PLU file F1 may be stored in the commodity reading apparatus 101 in addition to being stored in the POS terminal 11, or may be stored on a communicable server apparatus.

  The program executed in each device of the present embodiment has a module configuration including the above-described functional configuration. As actual hardware, the CPU (processor) reads out the program from the ROM and executes the program. The configuration is loaded on the main storage device and generated on the main storage device.

  DESCRIPTION OF SYMBOLS 1 ... Checkout system, 11 ... POS terminal, 21 ... Drawer, 22 ... Keyboard, 22d ... Numeric keypad, 22e ... Temporary fastening key, 22f ... Fastening key, 23 ... Display device, 23a ... Display surface, 24 ... Display device for customers 24a ... display surface, 25 ... communication interface, 26 ... touch panel, 51 ... checkout stand, 60 ... microcomputer, 61 ... CPU, 62 ... ROM, 63 ... RAM, 64 ... HDD, 65 ... connection interface, 66 ... printer DESCRIPTION OF SYMBOLS 101 ... Merchandise reading apparatus 102 ... Housing 103 ... Reading window 104 ... Display / operation part 105 ... Touch panel 106 ... Display device 107 ... Keyboard 108 ... Card reading groove 109 ... Display device for customer 110 ... commodity reading part, 151 ... counter stand, 152 ... consignment surface, 15 ... shopping basket, 153a ... first shopping basket, 153b ... second shopping basket, 160 ... microcomputer, 161 ... CPU, 162 ... ROM, 163 ... RAM, 164 ... imaging unit, 165 ... audio output unit, 175, 176: Connection interface, 611: Sales registration unit, 1611 ... Image capture unit, 1612 ... Product detection unit, 1613 ... Similarity calculation unit, 1614 ... Product selection unit, 1615 ... Registered product notification unit, B, B11, B12 ... Select button, F1 ... PLU file, G ... Product, G10-G30 ... Select screen, G31-G42 ... Icon image, H ... Hand, PR ... Program, R ... Reading area, SC ... Store computer, T1-T3 ... Product list Table, T11-T13 ... Products

JP 2007-225793 A

Claims (6)

Image capturing means for capturing an image captured by the image capturing means;
For each product registered in the product master data, a similarity that calculates the degree of similarity between the feature quantity of the product stored in advance and the feature quantity of the object imaged by the imaging unit is calculated. Degree calculation means;
Based on the product master data, display a list of each product registered in the product master data on the display means, and if the display mode of products with high similarity in the list is different from other products Display control means for displaying
Selection accepting means for accepting selection of a product in the list;
An information processing apparatus comprising: The display control means makes the display mode different in at least one of a shaded display, a boxed line display, and an icon image display on the products in the list,
The information processing apparatus according to claim 1. When there are a plurality of products having a degree of similarity higher than a predetermined threshold in the list, the display control unit changes the display mode of the plurality of products for each similarity.
The information processing apparatus according to claim 1 or 2. The display control means displays a list of classifications to which the products with high similarity belong in the list for each classification of the products registered in the product master data.
The information processing apparatus according to any one of claims 1 to 3. Imaging means;
Image capturing means for capturing an image captured by the image capturing means;
For each product registered in the product master data, a similarity that calculates the degree of similarity between the feature quantity of the product stored in advance and the feature quantity of the object imaged by the imaging unit is calculated. Degree calculation means;
Based on the product master data, display a list of each product registered in the product master data on the display means, and display products with high similarity in the list different from other products Display control means for displaying in a mode;
Selection accepting means for accepting selection of a product in the list;
Sales registration processing means for performing sales registration processing of the selected product;
Store system equipped with. Computer
Image capturing means for capturing an image captured by the image capturing means;
For each product registered in the product master data, a similarity that calculates the degree of similarity between the feature quantity of the product stored in advance and the feature quantity of the object imaged by the imaging unit is calculated. Degree calculation means;
Based on the product master data, display a list of each product registered in the product master data on the display means, and display products with high similarity in the list different from other products Display control means for displaying in a mode;
Selection accepting means for accepting selection of a product in the list;
Program to make it function.

Images