JP2014160513A - Commodity recognition apparatus and method, and recognition dictionary adding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently perform addition of feature quantity data into a recognition dictionary by an easy operation.SOLUTION: A commodity recognition apparatus compares appearance feature quantity of a commodity extracted from a photographed image with feature quantity data in a recognition dictionary file, and extracts candidates of a commodity included in the image. When a plurality of commodities is extracted, the apparatus receives selection input of the commodity included in the image from the plurality of candidate commodities. When receiving the selection input of any commodity, the apparatus receives input of declaration whether addition of the feature quantity data stored in the recognition dictionary file is performed for the selected commodity. When receiving the input of declaration to perform the addition, the apparatus adds the extracted appearance feature quantity data into the recognition dictionary file as the feature quantity data for the commodity whose selection is received.

Description

  Embodiments described herein relate generally to a product recognition apparatus that recognizes a product from an image captured by an imaging unit, and a recognition dictionary addition program that adds feature amount data to a recognition dictionary used by the apparatus.

  The feature quantity of the article is extracted from the image data obtained by imaging the target article (object) with the imaging unit, and the similarity is calculated by comparing with the feature quantity data registered in the recognition dictionary. There is a technique for recognizing the type of the article. Such a technique for recognizing an article included in an image is called generic object recognition. Regarding this general object recognition technique, various recognition techniques are described in the following documents.

Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>
Further, a technique for performing general object recognition by dividing an image into regions for each object is described in the following document.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [Search August 10, 2010], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = repl & type = pdf>
In recent years, for example, in a retail store accounting system (POS system), a general object recognition technique is applied to a recognition device for a product purchased by a customer, in particular, a product without a barcode such as a vegetable or a fruit. It has been proposed. In this case, feature amount data representing surface information such as the appearance shape, color, pattern, and unevenness of the product to be recognized as parameters is stored in the recognition dictionary. The product recognition device extracts feature quantities on the appearance of the product from the image data of the product imaged by the imaging unit, and collates the feature quantities with the feature value data of each product registered in the recognition dictionary. Then, the product recognition apparatus outputs a product having similar feature data as a recognized product candidate.

JP 2010-237886 A

  In the general object recognition technique, it is known that recognition accuracy can be improved by adding feature amount data to the recognition dictionary, that is, adding feature amount data to the recognition dictionary based on the result of recognition processing. In particular, when the general object recognition technique is applied to a product recognition apparatus, the appearance of products such as vegetables and fruits changes depending on the production area, season, and the like. For this reason, the addition of the feature amount data to the recognition dictionary is very effective for improving the recognition accuracy.

  The problem to be solved by the present invention is to provide a product recognition apparatus capable of efficiently adding feature quantity data to a recognition dictionary by a simple operation and a recognition dictionary addition program implemented by the apparatus. is there.

  In one embodiment, the product recognition apparatus includes a feature amount extraction unit, a product candidate extraction unit, a product selection reception unit, an additional declaration reception unit, and an addition unit. The feature amount extraction unit extracts the appearance feature amount of the product included in the image from the image captured by the imaging unit. The product candidate extraction means includes the feature quantity data of the recognition dictionary file in which the feature quantity data representing the surface information of the product for each product to be recognized is stored as the feature data extracted by the feature quantity extraction means. Collation is performed and product candidates included in the image are extracted. The product selection receiving unit receives a selection input of a product included in the image from the plurality of products when the product candidate extracting unit recognizes a plurality of products as candidates for the product included in the image. When the addition declaration accepting unit accepts a selection input of any one of a plurality of products by the product selection accepting unit, the addition declaration accepting unit adds the feature amount data stored in the recognition dictionary file for the selected product. Accepts declaration input for execution. When the adding means receives the declaration input for executing the addition by the addition declaration receiving means, the feature quantity data for the product whose selection is received by the product selection receiving means is used as the data of the appearance feature quantity extracted by the feature quantity extracting means. As a recognition dictionary file.

1 is an external view of a store accounting system. The block diagram which shows the hardware constitutions of a scanner apparatus and a POS terminal. The schematic diagram which shows the structure of the dictionary data preserve | saved at a recognition dictionary file. The block diagram for demonstrating the function as a goods recognition apparatus. The schematic diagram which shows a memory area required in order to implement | achieve a recognition dictionary addition function. The flowchart which shows the principal part of the information processing procedure which CPU of a goods recognition apparatus performs according to a goods recognition program and a recognition dictionary addition program. The flowchart which shows the specific procedure of the recognition process shown by step ST5 of FIG. The figure which shows an example of a registration goods selection screen. The figure which shows an example of an addition propriety selection screen.

  Hereinafter, an embodiment according to a product recognition device will be described with reference to the drawings. This embodiment is a case where the scanner device 1 and the POS (Point Of Sales) terminal 2 for constructing a store accounting system of a retail store that handles fresh food as products are provided with a function as a product recognition device. is there.

  FIG. 1 is an external view of a store accounting system. This system includes a scanner device 1 as a registration unit for registering products purchased by a customer, and a POS (Point Of Sales) terminal 2 as a settlement unit for processing payment for the customer. The scanner device 1 is mounted on the accounting counter 3. The POS terminal 2 is installed on the cash register 4 via the drawer 5. The scanner device 1 and the POS terminal 2 are electrically connected by a communication cable (not shown).

  The scanner device 1 includes a keyboard 11, a touch panel 12, and a customer display 13. These display / operation devices (keyboard 11, touch panel 12, customer display 13) are attached to a thin rectangular housing 1A that constitutes the main body of the scanner device 1.

  The housing 1A incorporates an imaging unit 14 as imaging means. A rectangular reading window 1B is formed in front of the housing 1A. The imaging unit 14 includes a CCD (Charge Coupled Device) imaging device that is an area image sensor and its drive circuit, and an imaging lens for forming an image of the imaging region on the CCD imaging device. The imaging region refers to a region of a frame image that forms an image on the area of the CCD imaging device from the reading window 1B through the imaging lens. The imaging unit 14 outputs an image of the imaging region formed on the CCD imaging device through the imaging lens.

  The POS terminal 2 includes a keyboard 21, an operator display 22, a customer display 23, and a receipt printer 24 as devices necessary for payment.

  The checkout counter 3 has an elongated shape along the customer passage on the back side. The cash register 4 is placed substantially perpendicular to the transaction counter 3 on the front side of the end of the transaction counter 3 on the downstream side with respect to the moving direction of the customer moving along the transaction counter 3. The front side of the accounting counter 3 and the front side of the cash register 4 serve as a space for so-called cashiers in charge of accounting.

  In the approximate center of the accounting counter 3, the housing 1A of the scanner device 1 is erected with the keyboard 11, the touch panel 12, and the reading window 1B facing the cashier side on the front side. The customer display 13 of the scanner device 1 is attached to the housing 1A so as to face the customer passage.

  On the upstream side of the customer counter in the direction of customer movement across the scanner device 1 of the checkout counter 3 is a space for placing a shopping basket 6 containing unregistered products M to be purchased by the shopper. On the other hand, the downstream cargo receiving surface is a space for placing a shopping basket 7 into which the commodity M registered by the scanner device 1 is placed.

  FIG. 2 is a block diagram illustrating a hardware configuration of the scanner device 1 and the POS terminal 2. The scanner device 1 includes a scanner unit 101 and an operation / display unit 102. The scanner unit 101 includes a CPU (Central Processing Unit) 111 as a control unit main body. A ROM (Read Only Memory) 113 and a RAM (Random Access Memory) 114 are connected to the CPU 111 via a bus line 112 such as an address bus or a data bus. The ROM 113 stores a program executed by the CPU 111.

  In addition, the imaging unit 14 is connected to the bus line 112 via an input / output circuit (not shown). The keyboard 11, the touch panel 12 and the customer display 13 are connected to the bus line 112 via the connection interface 115 and the connection interface 116. The touch panel 12 includes a panel display unit 121 using, for example, a liquid crystal display, and a touch panel sensor 122 arranged on the screen of the display unit.

  The connection interface 116, the keyboard 11, the touch panel 12, and the customer display 13 constitute the operation / display unit 102. Each unit constituting the operation / display unit 102 is controlled not only by the CPU 111 of the scanner unit 101 but also by a CPU 201 of the POS terminal 2 described later.

  The POS terminal 2 also includes a CPU 201 as a control unit main body. The CPU 201 is connected to a ROM 203, a RAM 204, an HDD (Hard Disk Drive) device 205, a communication interface 206, and a connection interface 207 via a bus line 202. Further, the keyboard 21, the operator display 22, the customer display 23, the printer 24, and the drawer 5 are connected to the bus line 202 through input / output circuits (not shown).

  The communication interface 206 is connected to the store server 8 serving as the center of the store via a network such as a LAN (Local Area Network). With this connection, the POS terminal 2 can exchange data with the store server 8. Thus, the POS terminal 2 accesses the recognition dictionary file 9 and the product data file (not shown) possessed by the store server 8 and reads the data record stored in each file or updates (adds or changes) the data record. , Delete).

  The connection interface 207 is connected to both connection interfaces 115 and 116 of the scanner device 1 via a communication cable. With this connection, the POS terminal 2 receives information read by the scanner unit 101 of the scanner device 1. The POS terminal 2 transmits and receives data signals to and from the keyboard 11, touch panel 12, and customer display 13 that constitute the operation / display unit 102 of the scanner device 1.

  FIG. 3 is a schematic diagram showing the structure of dictionary data stored in the recognition dictionary file 9. As shown in FIG. 3, the recognition dictionary file 9 stores a plurality of feature amount data in association with a product ID and a product name for identifying a product for each product to be recognized. The feature amount data represents the feature amount on the appearance, which is the surface information (appearance shape, hue, pattern, unevenness, etc.) of the product identified by the corresponding product ID. Feature quantity data 0 to N when the product is viewed from various directions are stored. Note that the number (N + 1) of feature amount data for one product is not fixed. Further, the number (N + 1) of feature amount data varies depending on the product.

  A unique product ID is assigned to each product sold in the store. In the product data file, product information such as product name, unit price, and preset image is stored in association with the product ID of each product.

  FIG. 4 is a block diagram for explaining a function as a product recognition device constituted by the scanner device 1 and the POS terminal 2. This function includes a feature amount extraction unit 41, a product candidate extraction unit 42, a product selection reception unit 43, an addition declaration reception unit 44, and an addition unit 45.

  The feature quantity extraction unit 41 extracts feature quantities (appearance feature quantities) on the appearance such as the shape, surface color, pattern, and unevenness of the product included in the image from the image captured by the imaging unit 14. The product candidate extracting unit 42 sequentially compares the feature data of the appearance extracted by the feature amount extracting unit 41 with the feature amount data of each product stored in the recognition dictionary file 9, and stores the data for each product. Calculate similarity. The method of calculating the similarity is not particularly limited. For example, the similarity is calculated using a Hamming distance method that counts the number of characters that are different from each other in the corresponding positions of feature data having the same number of characters. May be. The product candidate extraction unit 42 extracts products whose similarity is higher than the reference as candidates for products included in the image. Note that the degree of similarity may be, for example, a degree of coincidence (a coincidence rate) indicating a degree of coincidence, a correlation value indicating how much correlation is performed, and the like. That is, the similarity may be a value obtained based on the feature amount of the image captured by the imaging unit 14 and the feature amount stored in the recognition dictionary file 9.

  When the product candidate extracting unit 42 recognizes a plurality of products as candidates for a product included in the image, the product selection receiving unit 43 receives a selection input of a product included in the image from the plurality of products. When the product selection receiving unit 43 receives a selection input of any one of a plurality of products, the additional declaration receiving unit 44 stores the feature amount data stored in the recognition dictionary file 9 for the selected product. Declaration input of whether to execute addition of is accepted. When the addition declaration accepting unit 44 accepts a declaration input indicating that the addition is to be performed, the adding unit 45 receives the appearance feature value data extracted by the feature value extracting unit 41 from the product selection accepting unit 43. Is added to the recognition dictionary file 9 as feature amount data. That is, the product recognition device (scanner device 1, POS terminal 2) has a recognition dictionary addition function for adding feature amount data to the recognition dictionary file 9 based on the result of recognition processing.

  FIG. 5 is a schematic diagram showing a memory area necessary for realizing the recognition dictionary addition function. The memory area 51 is a storage area for the additional execution flag F. The product recognition apparatus validates the recognition dictionary addition function when the additional execution flag F is on, and invalidates the function when the addition execution flag F is off. The memory area 52 is a table area in which a condition (confirmation condition) for confirming whether or not to add the recognition dictionary file 9 is set by the status (1 or 0). The confirmation conditions include “similarity difference” and “candidate rank”. When the status is “1”, the corresponding confirmation condition is valid. When the additional execution flag F is on, the status of either the confirmation condition “similarity difference” or “candidate rank” is “1”.

  The additional declaration receiving unit 44 includes the feature amount data stored in the product recognition dictionary file 9 for which the selection input is received by the product selection receiving unit 43 and the appearance feature amount data extracted by the feature amount extracting unit 41. The similarity between the feature quantity data stored in the recognition dictionary file 9 of another product extracted as a candidate by the product candidate extraction means 42 and the appearance feature quantity data extracted by the feature quantity extraction means 41 When the comparison is low, a declaration input as to whether or not to perform addition is accepted.

  Specifically, the addition is executed when the difference between the similarity of the product for which the selection input has been accepted and the highest similarity among the similarities of the other products recognized as candidates is equal to or greater than a predetermined value. Accepts declaration input of whether or not. Alternatively, whether the addition is executed when the similarity of the product for which the selection input has been received corresponds to a predetermined order other than the first in the order in which the similarities of all the products recognized as candidates are arranged in descending order Accept declarative input of no. The former additional declaration acceptance means is executed when the status for the confirmation condition “similarity difference” is “1”. The latter additional declaration acceptance means is executed when the status for the confirmation condition “candidate rank” is “1”.

  The memory areas 51 and 52 are formed in the RAM 204 of the POS terminal 2. Memory areas 51 and 52 may be formed in the RAM 114 of the scanner device 1. Whether to turn on the additional execution flag F in the memory area 51 is automatically or manually executed in the setting work of the POS terminal 2. Similarly, the setting of which status is set to “1” among the statuses for each confirmation condition in the memory area 51 is automatically or manually executed in the setting work of the POS terminal 2.

  FIG. 6 is a flowchart showing the main part of the information processing procedure executed by the CPU (CPU 111, CPU 201) of the product recognition device (scanner device 1, POS terminal 2) according to the product recognition program and the recognition dictionary addition program. In the figure, the process from step ST1 to step ST7 is a process according to the product recognition program, and the process from step ST8 to step ST14 is a process according to the recognition dictionary addition program.

  The product recognition program and the recognition dictionary addition program may be independent programs or a single program. The product recognition program and the recognition dictionary addition program are stored in the ROM 203 of the POS terminal 2. At least a part of the program may be stored in the ROM 113 of the scanner device 1.

  When the processing of FIG. 6 starts, the CPU (CPU 111 or CPU 201: the same applies hereinafter) outputs an imaging on signal to the imaging unit 14 (ST1). For example, the CPU 201 instructs the CPU 111 to turn on imaging, and the CPU 111 that has received this command outputs an imaging on signal to the imaging unit 14. In response to this imaging on signal, the imaging unit 14 starts imaging of the imaging region. Frame images of the imaging area captured by the imaging unit 14 are sequentially stored in the RAM 114.

  The CPU takes in the frame image data stored in the RAM 114 (ST2). Then, the CPU confirms whether or not a product is detected from this frame image (ST3). Specifically, the CPU extracts a contour line and the like from an image obtained by binarizing the frame image. Then, the CPU tries to extract the contour of the object displayed in the frame image. When the contour of the object is extracted, the CPU regards the image within the contour as a product.

  If no product is detected from the frame image (NO in ST3), the CPU captures the next frame image from RAM 114 (ST2). Then, the CPU confirms whether or not a product is detected from this frame image (ST3).

  When a product is detected from the frame image (YES in ST3), the CPU uses the image in the outline to determine the feature amount (appearance feature amount) such as the shape, surface color, pattern, and unevenness of the product. ) Is extracted (ST4: feature amount extraction means 41). The extracted appearance feature amount data is temporarily stored in the work area of the RAM 204.

  When the feature amount extraction is completed, the CPU executes a procedure recognition process specifically shown by the flowchart of FIG. 7 (ST5: product candidate extraction means 42). First, the CPU searches the recognition dictionary file 9 of the store server 8 (ST21). Then, the CPU reads a data record of one product (product ID, product name, multiple feature data) from the recognition dictionary file 9 (ST22).

  If the data record is read, the CPU calculates a similarity indicating how similar the appearance feature data extracted in the process of step ST4 is to the feature data of the record (ST23). ). It can be said that the similarity is higher as the value is higher. In the present embodiment, assuming that the upper limit of similarity is “100”, the similarity of feature amount data is calculated for each product.

  The CPU confirms whether the similarity is higher than a predetermined reference threshold (ST24). The reference threshold is the lower limit of the similarity of products that should be left as registered product candidates. As described above, when the upper limit value of the similarity is “100”, the reference threshold value is set to “1/5”, for example, “20”. When the degree of similarity is higher than the reference threshold (YES in ST24), the CPU stores the product ID and product name of the data record, the appearance feature amount data extracted in step ST4, and step ST23. The similarity calculated in the process is stored as a registered product candidate in a predetermined area of the RAM 204 (ST25). On the other hand, when the similarity does not exceed the reference threshold value (NO in ST24), the CPU does not execute the process of step ST25.

  Thereafter, the CPU confirms whether or not an unprocessed data record exists in the recognition dictionary file 9 (ST26). If it exists (YES in ST26), the CPU returns to the process of step ST22. That is, the CPU further reads an unprocessed data record from the recognition dictionary file 9 and executes the processes of steps ST23 to ST26.

  In this way, when the processes of steps ST23 to ST26 are executed for the data records of all the products stored in the recognition dictionary file 9 (NO in ST26), the recognition process ends. When the recognition process is completed, the CPU confirms whether or not there is a registered commodity candidate (ST6).

  If no product data (product code, product name, appearance feature, similarity) that is a registered product candidate is stored in a predetermined area of the RAM 204, there is no registered product candidate. In this case (NO in ST6), the CPU returns to the process in step ST2. That is, the CPU fetches the next frame image data from the RAM 114. Then, the CPU executes the processes of steps ST3 to ST6 for this image data.

  On the other hand, if even one piece of product data (product code, product name, appearance feature, similarity) as a registered product candidate is stored in a predetermined area of the RAM 204, there is a registered product candidate. In this case (YES in ST6), the CPU confirms whether or not the registered product can be automatically set (ST7). Specifically, the CPU checks whether or not there is only one piece of data whose similarity exceeds a predetermined determination threshold among the product data that are registered product candidates. The decision threshold is a value sufficiently larger than the reference threshold. If the upper limit value of the similarity is “100” as described above, the decision threshold is set to a value slightly more than half, for example, “60”.

  When only one product whose similarity exceeds the determination threshold among registered product candidates, this product is automatically determined as a registered product. In other cases, that is, in the case where there is no product that has a similarity that exceeds the determination threshold, or there are two or more products, the registered product is not determined. When the registered product is determined (YES in ST7), the CPU jumps to the processing after ST8, that is, the processing routine of the recognition dictionary addition program, and the next processing, that is, the registration processing of the automatically determined product. Move to routine.

  On the other hand, when the registered product is not determined (NO in ST7), the CPU displays the registered product selection screen 60 on the touch panel 12 (ST8: product selection receiving means 43).

  An example of the registered product selection screen 60 is shown in FIG. As illustrated, the registered product selection screen 60 is divided into a captured image display region 61 and a candidate product display region 62. In addition, an “other” button 63 is displayed on the registered product selection screen 60. In the captured image display area 61, the frame image captured in the process of step ST2 is displayed. The candidate product display area 62 is further subdivided into three regions 621, 622, and 623, and preset images of products that are registered product candidates are displayed in descending order of similarity from the top of the screen.

  Incidentally, on the initial screen, the preset images of the products having the first to third similarities are displayed in order from the top of the screen in the candidate product display area 62 (621, 622, 623). In this state, when the “others” button 63 is touch-operated, the candidate product display area 62 is switched to the preset images of products whose similarity is from the 4th place to the 6th place. Thereafter, each time the “others” button 63 is touched, the image in the region 62 is switched to a preset image of a product with a lower similarity. When the “other” button 63 is touched once, an image of the “return” button is displayed on the registered product selection screen 60. When this “return” button is touch-operated, the screen of the touch panel 12 returns to the previous registered product selection screen 60.

  The user who holds the registered product over the reading window 1B searches for the registered product in the candidate product display area 62. If the registered product is found, the user touches the area 621, 622, or 623 where the preset image of the product is displayed.

  The CPU waits for the candidate product display area 62 to be touched. If the candidate product display area 62 is touched, the CPU confirms the order of similarity of the products for which the preset image is displayed in the touch area (ST9). When the product with the highest similarity is selected (YES in ST9), the CPU jumps to the processing after ST10 and proceeds to the next processing, that is, the registration processing routine for the product with the highest similarity. To do.

  On the other hand, when a product with the second highest similarity is selected (NO in ST9), the CPU checks the additional execution flag F (ST10). When the additional execution flag F is off, the recognition dictionary is not added. In this case (NO in ST10), the CPU jumps to the processing after ST11 and proceeds to the next processing, that is, the registration processing routine for the selected product.

  On the other hand, if the additional execution flag F is on, the CPU determines whether confirmation of additional execution is necessary (ST11). That is, the CPU checks the memory area 52 and recognizes a confirmation condition whose status is “1”. When the confirmation condition is “similarity difference”, the CPU calculates the difference between the similarity of the selected product and the first similarity, and confirms whether the difference value exceeds a predetermined threshold value. To do. The CPU determines that it is necessary to confirm with the user when the difference value exceeds a predetermined threshold value, and determines that it is not necessary to confirm with the user when the difference value is equal to or less than the predetermined threshold value.

  On the other hand, when the confirmation condition is “candidate rank”, the CPU confirms whether or not the similarity rank of the selected product is equal to or lower than a predetermined threshold rank (eg, third rank) lower than the first rank. To do. Then, the CPU determines that it is necessary to confirm with the user when the similarity rank is lower than the threshold rank, and determines that it is not necessary to confirm with the user when the similarity rank is higher than the threshold rank.

  If it is determined in step ST11 that confirmation is necessary (YES in ST11), the CPU displays an addition availability selection screen 70 on the touch panel 12 (ST12: addition declaration receiving means 44).

  An example of the addability selection screen 70 is shown in FIG. As shown in the drawing, the addition availability selection screen 70 is divided into a captured image display area 71 and a selected product display area 72. In addition, a “Yes” button 73 and a “No” button 74 are displayed on the addition availability selection screen 70. In the captured image display area 71, the frame image captured in the process of step ST2 is displayed. In the selected product display area 72, a preset image of the product selected on the registered product selection screen 60 is displayed. FIG. 9 is an addability selection screen 70 when the product “pear” whose preset image is displayed in the area 622 is selected on the registered product selection screen 60 of FIG. 8. Note that the position of the selected product display area 72 may or may not match the candidate product display area 62 in which the preset image of the product selected on the registered product selection screen 60 is displayed as shown in FIG. It may be. When it is displayed at the same position as the candidate product display area 62 where the preset image is displayed, it can be seen at a glance that it has been selected.

  The user touches the “Yes” button 73 when executing the addition of the feature amount data to the recognition dictionary. On the other hand, for example, when the product is selected and mistaken and the addition is not executed, the “do not” button is touched.

  The CPU waits for either the “Yes” button 73 or the “No” button 74 to be touched (ST13). If “NO” button 74 is touched (NO in ST13), the CPU jumps to the process in step ST13 and proceeds to the next process. However, since the selected product is an error, the product registration process routine is not executed.

  When “Yes” button 73 is touched (YES in ST13), or when it is determined that confirmation is unnecessary in the process of step ST11 (NO in ST11), the CPU determines the characteristics of recognition dictionary file 9. Addition of quantity data is executed (ST14: addition means 45). That is, the CPU reads out the product code and the data of the appearance feature value from the product data (product code, product name, appearance feature value, similarity) of the selected product. Then, the CPU accesses the recognition dictionary file 9 and adds the appearance feature value data as new feature value data to the data record including the product code. Thereafter, the CPU proceeds to the next processing, that is, the registration processing routine of the selected product.

  As described above, in the store accounting system of the present embodiment, when a user holds a product over the reading window 1B of the scanner device 1, the product is photographed by the imaging unit 14. The feature similarity is calculated for each product from the appearance feature data of the product extracted from the product image and the feature data of each product registered in the recognition dictionary file 9. . Then, registered product candidates are determined in descending order of similarity, and the list is displayed on the touch panel 12. Therefore, the user selects a corresponding product from the registered product candidates. Then, in the store accounting system, the sales data of the selected product is registered.

  Here, when the selected product is the product with the highest degree of similarity, the feature value data of the product registered in the recognition dictionary file 9 approximates the appearance feature value of the product. . Therefore, it is not necessary to add feature amount data to the recognition dictionary file 9. On the other hand, when the similarity of the selected product is lower than the similarity of the other products not selected, it is necessary to add feature amount data to the recognition dictionary file 9.

  In the store accounting system of the present embodiment, when the selected product is the product with the highest degree of similarity, the addition of the feature amount data to the recognition dictionary file 9 is not executed. When the selected product is a product other than the highest similarity, addition of feature amount data to the recognition dictionary file 9 is executed. However, when the addition execution flag F is reset to “0”, addition of feature amount data to the recognition dictionary file 9 is not permitted, and therefore a product other than the first similarity item is selected. However, the addition of the feature amount data to the recognition dictionary file 9 is not executed.

  If “similarity difference” is set as the confirmation condition, that is, if the status corresponding to the confirmation condition “similarity difference” is set to “1”, the similarity of the selected product is The difference with the highest similarity among the products recognized as registered product candidates is calculated. Then, it is determined whether or not this difference exceeds a predetermined threshold value. When the difference does not exceed the threshold value, that is, when the degree of similarity of the selected product is relatively high, the feature amount data is added to the recognition dictionary file 9.

  On the other hand, when the difference exceeds the threshold value, that is, when the similarity of the selected product is low, the user may have selected the wrong product. Therefore, an addability selection screen 70 is displayed on the touch panel 14. The user who has confirmed the addition permission / inhibition selection screen 70 touches the “Yes” button 73 when there is no error in product selection, and touches the “No” button 74 when there is an error. As a result, when there is no error in product selection, addition of feature amount data to the recognition dictionary file 9 is executed, but when there is an error, addition of feature amount data to the recognition dictionary file 9 is not executed.

  If “candidate rank” is set as the confirmation condition, that is, if the status corresponding to the confirmation condition “candidate rank” is set to “1”, the similarity of the selected product is registered. It is confirmed how many products are recognized as product candidates and arranged in descending order. Then, it is determined whether it is higher or lower than a predetermined threshold order lower than the first place. As a result, if it is higher than or equal to the threshold ranking, that is, if the degree of similarity of the selected product is relatively high, the feature amount data is added to the recognition dictionary file 9.

  On the other hand, if it is lower than the threshold ranking, that is, if the similarity of the selected product is low, the user may have selected the wrong product. Therefore, an addability selection screen 70 is displayed on the touch panel 14. The user who has confirmed the addition permission / inhibition selection screen 70 touches the “Yes” button 73 when there is no error in product selection, and touches the “No” button 74 when there is an error. As a result, when there is no error in product selection, addition of feature amount data to the recognition dictionary file 9 is executed, but when there is an error, addition of feature amount data to the recognition dictionary file 9 is not executed.

  As described above, according to the present embodiment, it is possible to efficiently add feature amount data to the recognition dictionary with a simple operation.

The present invention is not limited to the above embodiment.
For example, in the above-described embodiment, two types of “similarity difference” and “candidate rank” are shown as confirmation conditions for determining whether or not to add feature amount data to the recognition dictionary file 9. It may be. Alternatively, other conditions may be adopted.

  In addition, the addition declaration receiving unit 44 displays button images 73 and 74 for prompting a declaration input as to whether or not to perform addition on the touch panel 12 and receives a declaration input as to whether or not to perform the addition. It is not limited to. For example, a declaration input indicating whether or not to perform addition may be received by inputting a predetermined key of the keyboard 11. The same applies to the merchandise selection receiving means 43, and is not limited to the input operation on the registered merchandise selection screen 60. In short, it is sufficient that the user can select from a plurality of products recognized as registered product candidates.

  In the above-described embodiment, the scanner device 1 and the POS terminal 2 have a function as a product recognition device. However, the scanner device 1 or the POS terminal 2 may have a function as a product recognition device. Good. Alternatively, the function of the product recognition device may be provided in a device in which the scanner device 1 is integrated in the POS terminal 2.

  Further, a product recognition apparatus is configured from the scanner device 1, the POS terminal 2, and the server, and the product recognition function is provided in the server, that is, the product recognition program is provided in the server. May be executed. Specifically, an image of a product is captured by a scanner, and the product image captured by the scanner is transmitted to a server. The server collates the feature amount of the product image with the feature amount stored in the recognition dictionary, executes a product candidate recognition process, and outputs the recognition result at this server to the POS terminal 2.

  In the embodiment, a control program for realizing the functions of the invention is recorded in advance in a ROM which is a program storage unit in the apparatus. However, the present invention is not limited to this, and a similar program may be downloaded from the network to the apparatus. Alternatively, a similar program recorded on the recording medium may be installed in the apparatus. The recording medium may be in any form as long as it can store a program such as a CD-ROM or a memory card and can be read by the apparatus. Further, the function obtained by installing or downloading the program may be realized in cooperation with an OS (operating system) in the apparatus.

  In addition, although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Scanner apparatus, 2 ... POS terminal, 9 ... Recognition dictionary file, 11 ... Keyboard, 12 ... Touch panel, 14 ... Imaging part, 41 ... Feature quantity extraction means, 42 ... Product candidate extraction means, 43 ... Product selection reception means, 44 ... Additional declaration receiving means, 45 ... Additional means.

Embodiments described herein relate generally to a product recognition apparatus and method for recognizing a product from an image captured by an imaging unit, and a recognition dictionary addition program for adding feature data to a recognition dictionary used by the product recognition apparatus.

The problem to be solved by the present invention is to provide a product recognition apparatus and method capable of efficiently adding feature data to a recognition dictionary with a simple operation, and a recognition dictionary addition program implemented by the apparatus. Is.

In one embodiment, the product recognition apparatus includes a feature amount extraction unit, a recognition unit, a reception unit, and an addition unit. Feature extraction means extracts the appearance feature amount of the product contained from the shooting image image to the image. The recognizing unit collates the appearance feature amount data extracted by the feature amount extracting unit with the feature amount data of the recognition dictionary file in which the feature amount data representing the surface information of the product is stored for each product to be recognized. Te, to recognize the candidate of the products included in the image. When the product included in the image is not specified from the candidates recognized by the recognition unit, the receiving unit receives an input for specifying the product included in the image from the candidates . The adding unit adds the appearance feature amount data extracted by the feature amount extracting unit to the recognition dictionary file as feature amount data for the product specified by the input received by the receiving unit .

Claims (6)

Feature quantity extraction means for extracting the appearance feature quantity of the product included in the image from the image taken by the imaging means;
The feature amount data extracted by the feature amount extraction means is collated with the feature amount data in the recognition dictionary file in which feature amount data representing the surface information of the product is stored for each product to be recognized, Product candidate extraction means for extracting product candidates included in the image;
When a plurality of product candidates are extracted by the product candidate extraction unit, a product selection receiving unit that receives a selection input of a product included in the image from the plurality of product candidates;
When the selection input of any one of the plurality of products is received by the product selection receiving unit, the feature amount data stored in the recognition dictionary file is added to the selected product. An additional declaration receiving means for receiving a declaration input of whether or not,
When a declaration input for executing addition is received by the additional declaration receiving unit, the feature data for the product whose selection is received by the product selection receiving unit is used as the data of the appearance feature amount extracted by the feature amount extracting unit. Adding means to add to the recognition dictionary file as
A product recognition apparatus comprising:   The additional declaration receiving means is a similarity between the feature value data stored in the recognition dictionary file of the product for which the selection input is received by the product selection receiving means and the appearance feature value data extracted by the feature value extracting means. Is compared with the similarity between the feature amount data stored in the recognition dictionary file of other products extracted as candidates by the product candidate extraction unit and the appearance feature amount data extracted by the feature amount extraction unit. The product recognition apparatus according to claim 1, wherein when it is low, a declaration input as to whether or not to perform the addition is accepted.   The additional declaration receiving unit is configured such that the similarity of the product for which selection input is received by the product selection receiving unit and the highest similarity among the similarities of the other products extracted as candidates by the product candidate extracting unit The product recognition apparatus according to claim 2, wherein when the difference is equal to or greater than a predetermined value, a declaration input as to whether or not to perform the addition is accepted.   The addition declaration accepting unit is configured such that the similarity of the products for which selection input is received by the product selection receiving unit is 1 in the order in which the similarities of all products extracted as candidates by the product candidate extracting unit are arranged in descending order. 3. The product recognition apparatus according to claim 2, wherein a declaration input as to whether or not to execute the addition is accepted when the order corresponds to a predetermined order other than the order. Equipped with a touch panel,
5. The product recognition apparatus according to claim 1, wherein the addition declaration receiving unit displays a button image for prompting a declaration input as to whether or not to perform the addition on the touch panel. The appearance feature amount of the product included in the image is extracted from the image captured by the imaging means, and the extracted appearance feature amount data is the feature amount data representing the surface information of the product for each product to be recognized. In the product recognition device that extracts the product candidates included in the image by comparing with the feature amount data of the recognition dictionary file to be stored,
A function of accepting selection input of a product included in the image from the plurality of product candidates when a plurality of products is recognized as a product candidate included in the image;
When a selection input of any one of the plurality of products is received, a declaration input indicating whether or not to add the feature amount data stored in the recognition dictionary file for the selected product A function to accept
When receiving a declaration input for performing the addition, a function of adding appearance feature data of a product included in the image as feature data for the product that has received the selection input of the recognition dictionary file;
Recognition dictionary addition program to realize.

Images