JP6412911B2 - Information processing apparatus and program - Google Patents

Description

  Embodiments described herein relate generally to an information processing apparatus and a program.

  2. Description of the Related Art Conventionally, there is a traceability system that enables consumers to browse traceability from production to retail of products such as agricultural products and processed foods. In such a traceability system, the product origin, production / manufacturing method, etc. are recorded in association with the corresponding product identification information, and the product is read by reading the identification information from a barcode or wireless IC tag attached to the product. It is possible to specify the production area and production / manufacturing method.

  However, in the conventional technology, since it is necessary to directly add a barcode or a wireless IC tag to each product in order to obtain additional information such as the production area, the amount of work and cost required for attaching them increases as the number of products increases. There is a problem that increases.

The information processing apparatus according to the embodiment includes first storage means, product type identification means, detection means, individual identification means, and output means. The first storage means includes a product code indicating the product type, a reference image captured for each product belonging to the product code type so as to include a symbol attached to the product, and the reference image. The characteristic information unique to the individual is associated and stored. The product type identification means is a feature of an area other than the area where the feature amount of the captured image obtained by imaging the product to be identified and the symbol included in the reference image of each product stored in the first storage means. The product code of the product to be identified is identified based on the similarity between the two feature values. The detection means detects the symbol attached to the product from the captured image of the product for which the product code is identified by the product type identification means. When the symbol is detected by the detection unit, the individual identification unit is a region in which the symbol included in each of the reference images stored in association with the product code identified by the type identification unit exists. The feature amount is compared with the feature amount of the captured image, and an individual reference image corresponding to the product to be identified is identified based on the similarity between both feature amounts. The output means outputs the characteristic information stored in association with the reference image identified by the individual identification means .

FIG. 1 is a perspective view illustrating an external configuration of a checkout system to which an information processing apparatus and a program according to an embodiment are applied. FIG. 2 is a block diagram illustrating a hardware configuration of the POS terminal and the commodity reading apparatus. FIG. 3 is a diagram schematically illustrating an example of the data configuration of the PLU file. FIG. 4 is a diagram schematically illustrating an example of the data configuration of the product characteristic file. FIG. 5 is a block diagram illustrating functional configurations of the POS terminal and the commodity reading apparatus. FIG. 6 is a diagram illustrating an example of an imaging condition instruction screen displayed on the display device of the commodity reading apparatus. FIG. 7 is a diagram illustrating an example of a product information screen displayed on the display device of the product reading apparatus. FIG. 8 is a flowchart showing a procedure of processing operations in the commodity reading apparatus. FIG. 9 is a flowchart showing the flow of processing operations in the POS terminal.

  Hereinafter, an information processing apparatus and a program according to the present embodiment will be described with reference to the drawings, taking a checkout system as an example. The store system is a checkout system (POS system) provided with a POS terminal that performs registration and settlement of commodities related to one transaction. This embodiment is an application example to a checkout system introduced in a store such as a supermarket.

  FIG. 1 is a perspective view showing an external configuration of the checkout system 1. As shown in FIG. 1, the checkout system 1 includes a POS terminal 11 that performs registration and settlement of commodities related to one transaction. The POS terminal 11 is placed on the upper surface of the drawer 21 on the checkout table 51. The drawer 21 is controlled by the POS terminal 11 for the opening operation.

  On the upper surface of the POS terminal 11, a keyboard 22 that is pressed down by an operator (a store clerk) is arranged. A display device 23 that displays information toward the operator is provided behind the keyboard 22 when viewed from the operator who operates the keyboard 22. The display device 23 displays information on the display surface 23a. A touch panel 26 is laminated on the display surface 23a. A customer display device 24 is erected so as to be rotatable further behind the display device 23. The customer display device 24 displays information on the display surface 24a.

  The customer display device 24 shown in FIG. 1 has the display surface 24a facing the front side in FIG. 1, but the customer display device 24 is rotated so that the display surface 24a faces the back side in FIG. Thus, the customer display device 24 displays information to the customer.

  A horizontally long table-shaped counter table 151 is arranged so as to form an L shape with the checkout table 51 on which the POS terminal 11 is placed. A load receiving surface 152 is formed on the upper surface of the counter table 151. A shopping basket 153 for storing the product G is placed on the cargo receiving surface 152. The shopping basket 153 may be divided into a first shopping basket 153a brought in by a customer and a second shopping basket 153b placed at a position sandwiching the commodity reading device 101 from the first shopping basket 153a. it can.

  On the receiving surface 152 of the counter stand 151, the commodity reading apparatus 101 connected to the POS terminal 11 so as to be able to transmit and receive data is installed. The commodity reading apparatus 101 includes a thin rectangular housing 102.

  A reading window 103 is disposed in front of the housing 102. A display / operation unit 104 is attached to the upper portion of the housing 102. The display / operation unit 104 is provided with a display device 106 on which a touch panel 105 is laminated. A keyboard 107 is disposed on the right side of the display device 106. A card reading groove 108 of a card reader (not shown) is provided on the right side of the keyboard 107. A customer display device 109 for providing information to the customer is installed on the back left side of the display / operation unit 104 as viewed from the operator.

  Such a product reading apparatus 101 includes a product reading unit 110 (see FIG. 2). The merchandise reading unit 110 has an imaging unit 164 (see FIG. 2) disposed behind the reading window 103.

  In the first shopping basket 153a brought in by the customer, a product G related to one transaction is stored. The commodity G in the first shopping basket 153a is moved to the second shopping basket 153b by an operator who operates the commodity reading apparatus 101. In this movement process, the product G is directed to the reading window 103 of the product reading apparatus 101. At this time, the imaging unit 164 (see FIG. 2) arranged in the reading window 103 images the product G.

  FIG. 2 is a block diagram illustrating a hardware configuration of the POS terminal 11 and the commodity reading apparatus 101. The POS terminal 11 includes a microcomputer 60 as an information processing unit that executes information processing. The microcomputer 60 is configured by connecting a ROM (Read Only Memory) 62 and a RAM (Random Access Memory) 63 to a CPU (Central Processing Unit) 61 that executes various arithmetic processes and controls each unit.

  The drawer 61, the keyboard 22, the display device 23, the touch panel 26, and the customer display device 24 are all connected to the CPU 61 of the POS terminal 11 through various input / output circuits (all not shown). .

  The keyboard 22 includes a numeric key 22d, a temporary fastening key 22e, and a fastening key 22f on which numbers such as “1”, “2”, “3”,...

  An HDD 64 (Hard Disk Drive) is connected to the CPU 61 of the POS terminal 11. The HDD 64 stores programs and various files. All or a part of the programs and various files stored in the HDD 64 are copied to the RAM 63 and sequentially executed by the CPU 61 when the POS terminal 11 is activated.

  Examples of programs stored in the HDD 64 include a program PR for merchandise sales data processing, which will be described later. Examples of files stored in the HDD 64 include a PLU file F1 and a product characteristic file F2. The PLU file F1 and the product characteristic file F2 are configured to be readable (referenced) by the product reading apparatus 101 via a connection interface 65 described later.

  The PLU file F1 is a data file in which an association between information related to sales registration of the product G and an image of the product G is set.

  FIG. 3 is a diagram schematically illustrating an example of the data configuration of the PLU file F1. As shown in the figure, the PLU file F1 captures a product code uniquely assigned to each product G, product information about the product such as the product classification, product name, and unit price to which the product G belongs, and the product G. The product image (hereinafter referred to as the first product image) and the flag information indicating whether or not product characteristic information described later is attached to the product G are registered in association with each other. In FIG. 3, the value “1” of the flag information indicates that product characteristic information is attached, and the value “0” of the flag information indicates that no product characteristic information is attached.

  The product characteristic file F2 is a data file in which an association between the product G and the product characteristic information is set.

  FIG. 4 is a diagram schematically illustrating an example of the data configuration of the product characteristic file F2. As shown in the figure, in the product property file F2, the product code of the product G, the product property information of the product G, and each product property information are stored for each product G to which the product property information is attached. A product characteristic code to be identified and a product image obtained by imaging the product G (hereinafter referred to as a second product image) are registered in association with each other.

  In the product property file F2, the product code of each product G that is the supplementary target of the product property information among the products G registered in the PLU file F1 is registered. The product characteristic information includes, for example, traceability information relating to the production history of the product G, such as the production area, grade, sugar content, producer, and harvest date. A unique product characteristic code is assigned to each product characteristic information. In this embodiment, it is assumed that a predetermined symbol (hereinafter referred to as a characteristic symbol) such as a seal such as a logo mark or a branding mark is added in advance to the surface of the product G that is an ancillary object of the product characteristic information. .

  As the second product image, the product image of the product G imaged under a predetermined imaging condition is registered. In the present embodiment, as shown in FIG. 4, imaging is performed centering on the characteristic symbol CS added to the product G, and a part of the product G captured centering on the characteristic symbol CS is represented. The product image is registered as the second product image.

  Here, the first product image in the PLU file F1 is used for identifying (recognizing) the type of the product G (for example, apple (Fuji), apple (Wanglin), etc.), while the first product image in the product characteristic file F2. The two product images are used to identify (recognize) each individual product G of the same product type. Therefore, the first product image is an image obtained by capturing an arbitrary product G in each product type, and the second product image is individually captured for each product G under the same imaging conditions centering on the characteristic symbol. It has become an image. In this embodiment, it is possible to narrow down the individual product G from the product G type by recognizing the product G using these two product images in stages.

  Note that the data structures of the PLU file F1 and the product property file F2 are not limited to the examples in FIGS. For example, in FIG. 3 and FIG. 4, the first product image and the second product image are stored as images, but instead of this, feature amounts (color shades) read from each of the first product image and the second product image. Or a surface irregularity state) may be stored.

  Returning to FIG. 2, a communication interface 25 for executing data communication with the store computer SC is connected to the CPU 61 of the POS terminal 11 via an input / output circuit (not shown). The store computer SC is installed in a store backyard or the like. The HDD (not shown) of the store computer SC stores a PLU file F1 and a product characteristic file F2 distributed to the POS terminal 11.

  Further, the CPU 61 of the POS terminal 11 is connected to a connection interface 65 that enables data transmission / reception with the commodity reading apparatus 101. The product reading apparatus 101 is connected to the connection interface 65. The CPU 61 of the POS terminal 11 is connected to a printer 66 that prints on receipts. The POS terminal 11 prints the transaction content of one transaction on the receipt under the control of the CPU 61.

  The commodity reading apparatus 101 also includes a microcomputer 160. The microcomputer 160 is configured by connecting a ROM 162 and a RAM 163 to a CPU 161 via a bus. The ROM 162 stores a program executed by the CPU 161. The CPU 161 is connected to an imaging unit 164 and an audio output unit 165 via various input / output circuits (none of which are shown). The operations of the imaging unit 164 and the audio output unit 165 are controlled by the CPU 161. The display / operation unit 104 is connected to the POS terminal 11 via the connection interface 176. The operation of the display / operation unit 104 is controlled by the CPU 61 of the POS terminal 11.

  The imaging unit 164 is a color CCD image sensor, a color CMOS image sensor, or the like, and is an imaging unit that performs imaging from the reading window 103 under the control of the CPU 161. For example, the imaging unit 164 captures a 30 fps moving image. Frame images (captured images) sequentially captured at a predetermined frame rate by the imaging unit 164 are stored in the RAM 163.

  The audio output unit 165 is an audio circuit and a speaker for generating a preset warning sound or the like. The sound output unit 165 performs notification by sound such as a warning sound under the control of the CPU 161.

  Furthermore, a connection interface 175 that connects to the connection interface 65 of the POS terminal 11 and enables data transmission / reception with the POS terminal 11 is connected to the CPU 161. Product information and product characteristic information of the product G recognized from the captured image captured by the imaging unit 164 of the product reading device 101 under the control of the CPU 161 are output via the connection interface 175 and via the connection interface 65. Input to the POS terminal 11.

  Next, functional units of the CPU 161 and the CPU 61 that are realized by the CPU 161 and the CPU 61 sequentially executing programs will be described with reference to FIG.

  FIG. 5 is a block diagram illustrating functional configurations of the POS terminal 11 and the commodity reading apparatus 101. As illustrated in FIG. 5, the CPU 161 of the commodity reading apparatus 101 sequentially executes a program to obtain an image capturing unit 1611, a product type identifying unit 1612, an individual identifying unit 1613, a product information display unit 1614, and an information transmitting unit 1615. It has the function of.

  The image capturing unit 1611 outputs an imaging on signal to the imaging unit 164 and causes the imaging unit 164 to start an imaging operation. The image capturing unit 1611 sequentially captures captured images captured by the image capturing unit 164 and stored in the RAM 163 after the start of the image capturing operation. Captured images are captured by the image capture unit 1611 in the order stored in the RAM 163.

  The product type identifying unit 1612 identifies (recognizes) the product type of the product G from the captured image captured by the image capturing unit 1611 and identifies product information (product code) corresponding to the product type.

  Specifically, the product type identification unit 1612 detects the outline of all or part of the product G included in the captured image using a known pattern matching technique or the like. Next, the contour line extracted from the latest frame image is compared with the contour line extracted from the current frame image, and the portion where the change has occurred, that is, the reflection of the product G directed to the reading window 103 is shown. To detect.

  In addition, as another method of detecting the product G, it is determined whether or not a skin color area is detected from the captured image, and when the skin color area is detected, that is, when the reflection of the clerk's hand is detected. The contour line of the product G that is assumed to be held by the clerk's hand is detected. At this time, when the contour indicating the shape of the hand and the other contours are detected, since the store clerk's hand holds the product G, the reflection of the product G is detected.

  In addition, the product type identification unit 1612 reads the surface state (surface color, pattern (pattern), unevenness, etc.) of the product G included in the captured image as a feature amount, and stores each of the first items registered in the PLU file F1. By comparing with the feature value of one product image, the type of product G is identified. Here, the product type identification unit 1612 calculates the similarity between each first product image registered in the PLU file F1 and the product G included in the captured image by comparing both feature amounts. A first product image having a maximum similarity or exceeding a predetermined threshold is identified as a product image representing the type of product G. Then, the product type identification unit 1612 identifies the product code related to the identified first product image as the product information of the product G.

Note that identifying (recognizing) an object included in an image as described above is called generic object recognition. Regarding such general object recognition, various recognition techniques are described in the following documents.
Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

Further, techniques for performing general object recognition by dividing an image into regions for each object are described in the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [searched August 10, 2010], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>

  The individual identifying unit 1613 identifies an individual of the product whose product type has been identified by the product type identifying unit 1612, and identifies product characteristic information (product characteristic code) unique to the individual.

  Specifically, the individual identifying unit 1613 determines whether or not the product specifying information is attached to the product based on the flag information of the PLU file F1 associated with the product code specified by the product type identifying unit 1612. Here, when the value of the flag information indicates that the product identification information is attached, the individual identification unit 1613 uses a known pattern matching technique or the like to obtain the image of the product G included in the captured image. Detect characteristic symbols.

  When the characteristic symbol is detected from the captured image, the individual identifying unit 1613 determines that the product G is captured under the same imaging condition as the imaging condition of the second product image, and the product G centered on the characteristic symbol. From the captured image, the product G and the characteristic quantities (characteristics of the surface, pattern, unevenness, etc.) of the product G and characteristic symbols are read by the same method as the product type identification unit 1612. The individual identification unit 1613 compares the feature amount read from the captured image with the feature amount of each second product image in the product characteristic file F2 associated with the product code specified by the product type identification unit 1612, and the feature amount. Is identified as a product image representing an individual of the product G. Then, the individual identifying unit 1613 identifies the product characteristic code related to the identified second product image as the product characteristic information corresponding to the individual of the product G.

  When the characteristic symbol is not detected, the individual identification unit 1613 determines that the product G is not imaged under the same imaging condition as that of the second product image, and prompts the characteristic symbol to be directed to the imaging unit 164. By displaying (presenting) a screen (imaging condition instruction screen) on the display device 106, the operator is instructed to change the imaging condition of the product G.

  Here, FIG. 6 is a diagram illustrating an example of an imaging condition instruction screen displayed on the display device 106 under the control of the individual identification unit 1613. This figure shows an example in which a message “Please direct the characteristic symbol to the camera” for prompting the characteristic symbol to be directed to the imaging unit 164 is displayed in the area A11 in the screen. The area A12 is an area for displaying a captured image being captured by the imaging unit 164.

  By displaying the imaging condition instruction screen, the operator of the commodity reading apparatus 101 confirms the captured image displayed in the area A12 with the imaging direction (imaging condition) of the commodity G according to the message displayed in the area A11. The characteristic symbol that is changed and added to the product G is directed to the imaging unit 164. When the individual identification unit 1613 detects a characteristic symbol from the captured image captured by the image capturing unit 1611 due to a change in the imaging condition, the individual identification unit 1613 starts a process for specifying the product characteristic of the product G from the captured image.

  As described above, in the individual identification unit 1613 of the present embodiment, whether or not the imaging condition of the product G corresponds to the imaging condition when the second product image is captured based on the characteristic symbol added to the product G. When the characteristic symbol is not detected, the change in the imaging condition of the product G is instructed so as to approach the imaging condition when the second product image is captured. Can be improved. In the present embodiment, when the characteristic symbol is detected from the captured image, the individual of the product G is identified. However, the characteristic symbol is detected at the detection position of the characteristic symbol in the captured image, that is, approximately at the center of the captured image. It is good also as a form which determines whether it corresponds to imaging conditions including whether it exists.

  Returning to FIG. 5, the product information display unit 1614 displays the product information and the product characteristic information corresponding to the product code and the product characteristic code specified by the product type identification unit 1612 and the individual identification unit 1613, for the display device 106 and the customer use. A display is output on the display device 109.

  Specifically, the product information display unit 1614 reads product information corresponding to the product code of the product G specified by the product type identification unit 1612 from the PLU file F1, and displays the product information on the display device 106 or the customer display device 109. Further, when the product characteristic code of the product G is specified by the individual identification unit 1613, the product type identifying unit 1612 reads the product detailed information corresponding to the product characteristic code from the product characteristic file F2, and the product information of the corresponding product. At the same time, it is displayed on the display device 106 and the customer display device 109.

  Here, FIG. 7 is a diagram illustrating an example of a product information screen displayed on the display device 106 (customer display device 109) under the control of the product information display unit 1614. In the figure, a product image (first product image) of a product recognized by the product type identification unit 1612 is displayed in an area A21 in the screen. In the area A22 in the screen, product information and product characteristic information corresponding to the product code of the product G specified by the product type identification unit 1612 and the product characteristic code of the product G specified by the individual identification unit 1613 are displayed. Is displayed.

  In this way, by displaying the product information and product characteristic information of the product G on the display device 106 and the customer display device 109, both the operator and the customer can confirm the product G that has been read. . In FIG. 7, the first product image is displayed in the area A21. However, the present invention is not limited to this, and the second product image may be displayed, or the product image may not be displayed.

  Returning to FIG. 5, the information transmission unit 1615 notifies the POS terminal 11 of the product code of the product G specified by the product type identification unit 1612 and the product characteristic code of the product G specified by the individual identification unit 1613.

  Further, as shown in FIG. 5, the CPU 61 of the POS terminal 11 sequentially executes the program to thereby obtain an information receiving unit 611, a product information acquiring unit 612, a product characteristic information acquiring unit 613, a sales registering unit 614, an accounting processing unit. The function as 615 is provided.

  The information receiving unit 611 receives information transmitted from the information transmitting unit 1615 of the product reading apparatus 101, that is, information such as a product code and a product characteristic code.

  Based on the product code received by the information receiving unit 611, the product information acquisition unit 612 searches the PLU file F1 for product information of the product corresponding to the product code and acquires the product information.

  Based on the product characteristic code received by the information receiving unit 611, the product characteristic information acquisition unit 613 searches and acquires the product characteristic information of the product corresponding to the product characteristic code from the product characteristic file F2.

  The sales registration unit 614 calculates the sales price of the product from the unit price included in the product information acquired by the product information acquisition unit 612, the number of sales included in the information received by the information reception unit 611, and the like. The sales registration unit 614 registers sales by recording product information, product characteristic information, and the like of the product G read by the product reading apparatus 101 in a sales master file (not shown).

  The accounting processing unit 615 performs accounting processing for the product related to one transaction registered by the sales registration unit 614. Further, the accounting processing unit 615 associates the product information of each product related to one transaction registered by the sales registration unit 614 with the characteristic information of the corresponding product acquired by the product characteristic information acquisition unit 613, and displays the display device 23 or The data is displayed on the customer display device 24 or printed on a receipt using the printer 66.

  Next, the operation of the checkout system 1 having the above configuration will be described. First, the processing operation of the commodity reading apparatus 101 will be described. FIG. 8 is a flowchart showing a procedure of processing operations in the commodity reading apparatus 101.

  As illustrated in FIG. 8, when processing is started in response to the start of product registration by the POS terminal 11, the image capturing unit 1611 outputs an imaging on signal to the imaging unit 164 and performs imaging by the imaging unit 164. Start (step S11). Subsequently, the image capturing unit 1611 captures a frame image (captured image) captured by the image capturing unit 164 and stored in the RAM 163 (step S12).

  Next, the product type identification unit 1612 compares the feature amount of the product G in the captured image captured by the image capture unit 1611 with the feature amount of each first product image registered in the PLU file F1. The product code of the product G is specified (step S13). If there are a plurality of product type identification candidates, the product images and product information of the identification candidates are displayed (presented) on the display device 106, and the corresponding product (product type) is selected on the touch panel 105. It is good also as a form received from an operator via.

  Subsequently, the individual identification unit 1613 determines whether or not the product characteristic information is attached to the type of the product G based on the value of the flag information associated with the product code of the product G specified in step S13. (Step S14). Here, when the value of the flag information indicates that the product characteristic information is not attached (step S14; No), the individual identifying unit 1613 proceeds to step S15 without specifying the product characteristic.

  In step S15, the product information display unit 1614 reads product information corresponding to the product code of the product specified in step S13 from the PLU file F1, and displays it on the display device 106 or the customer display device 109 (step S15). Next, the information transmission unit 1615 transmits the product code or the like of the product specified in step S13 to the POS terminal 11 (step S16), and proceeds to step S22.

  In step S14, when the value of the flag information indicates that the product characteristic information is attached (step S14; Yes), the individual identifying unit 1613 uses the characteristic symbol from the captured image captured by the image capturing unit 1611. It is determined whether or not the characteristic symbol has been detected (step S17).

  Here, when it is determined that the characteristic symbol cannot be detected (step S17; No), the individual identification unit 1613 determines that the product G is not imaged under the same imaging condition as the imaging condition of the second product image. An imaging condition instruction screen (see FIG. 6) for instructing to direct the symbol to the imaging unit 164 is displayed on the display device 106 (step S18). Then, the process returns to step S17, and the display of the imaging condition instruction screen is continued until the characteristic symbol can be detected from the captured image.

  In step S17, when the characteristic symbol is detected from the captured image (step S17; Yes), the individual identification unit 1613 specifies the feature amount of the product G and the characteristic symbol image centered on the characteristic symbol, and the characteristic symbol in step S13. The individual product G is identified by comparing the feature quantity of each second product image in the product characteristic file F2 associated with the manufactured product code, and the product characteristic code corresponding to this individual is specified (step S19). ).

  Subsequently, the product information display unit 1614 displays the product information and product characteristic information corresponding to the product code of the product G specified in step S13 and the product characteristic code of the product G specified in step S19, in the PLU file F1. And it reads out from the product characteristic file F2 and displays it on the display device 106 or the customer display device 109 as shown in FIG. 7 (step S20). Next, the information transmission unit 1615 transmits the product code of the product G specified in step S13, the product characteristic code of the product G specified in step S19, and the like to the POS terminal 11 (step S21), and proceeds to step S22. .

  In step S <b> 22, the CPU 161 determines whether or not there is an end of business based on a product registration end notification from the POS terminal 11 (step S <b> 22). Here, when the business is continued (step S22; No), the CPU 161 returns the process to step S12 to continue the process. When the job is to be ended (step S22; Yes), the image capturing unit 1611 outputs the imaging off signal to the imaging unit 164, thereby ending the imaging by the imaging unit 164 (step S23) and ending the process. To do.

  Next, the processing operation of the POS terminal 11 will be described. FIG. 9 is a flowchart showing the flow of processing operations in the POS terminal 11.

  As shown in FIG. 9, in the POS terminal 11, the information receiving unit 611 receives information such as a product code and a product characteristic code from the product reading device 101 (step S31). The product information acquisition unit 612 acquires product information corresponding to the product code from the PLU file F1 based on the product code included in the information received in step S31 (step S32).

  The product characteristic information acquisition unit 613 determines whether or not the product characteristic code is included in the information received in step S31 (step S33). If the received information includes a product characteristic code (step S33; Yes), the product characteristic information acquisition unit 613 acquires product characteristic information corresponding to the product characteristic code from the product characteristic file F2 (step S33). S34), the process proceeds to step S35. If the received information does not include the product characteristic code (step S33; No), the process immediately proceeds to step S35.

  Thereafter, in the POS terminal 11, the sales registration unit 614 executes a process of registering the sales information of the product in the sales master file or the like based on the product information acquired in Step S32 (Step S35). If the sales number of the product is received in step S31, the sales price for the sales number is calculated by multiplying the unit price included in the product information by the sales number in step S35.

  Subsequently, in the POS terminal 11, the CPU 61 determines whether or not there is an end of business due to the end of sales registration or the like by an operation instruction of the keyboard 22 (such as a pressing operation of the closing key 22f) (step S36). Here, when the business is continued (step S36; No), the CPU 61 returns the process to step S31 and continues the process.

  In Step S36, when the business is ended (Step S36; Yes), the accounting processing unit 615 executes accounting processing for each commodity G related to one transaction (Step S37). Then, the accounting processing unit 615 displays the product information of each product in association with the product characteristic information on the display device 23 or the customer display device 24, or prints it on a receipt or the like using the printer 66. (Step S38), the process is terminated.

  As described above, according to the checkout system 1 of the present embodiment, the individual of the product G is identified from the captured image captured by the imaging unit 164, and the product characteristic information attached to the individual is obtained from the product characteristic file F2. Specify and output. Accordingly, product characteristic information unique to the product G can be obtained from the product G without using a barcode or a wireless IC tag, so that product property information can be efficiently acquired. In addition, by identifying the individual using the product G itself and specifying the product identification information of the individual, fraudulent acts such as counterfeiting and replacement that may occur when the product identification information is held using a wireless IC tag or the like Therefore, the reliability of the product identification information can be improved.

  As mentioned above, although embodiment of this invention was described, the said embodiment was shown as an example and is not intending limiting the range of invention. The above embodiment can be implemented in various other forms, and various omissions, replacements, changes, additions, and the like can be made without departing from the scope of the invention. Moreover, the said embodiment and its deformation | transformation are included in the range of the invention, the summary, and the invention described in the claim, and its equal range.

  For example, in the above embodiment, the first product image and the second product image are different product images. However, the present invention is not limited to this, and by registering the second product image in the PLU file F1, It is good also as a form to use. In the case of this embodiment, it is assumed that the product type identification unit 1612 reads the feature amount from a region other than the region where the characteristic symbol of the second product image exists and compares it with the feature amount of the product G. Specifically, in the case of the second product image illustrated in FIG. 4, since the characteristic symbol CS exists in the central portion of the image, the feature amount is read from a region other than the central portion (for example, the outer edge portion of the image). .

  Further, in the case of the above form using the second product image as the first product image, it is possible to switch the region where the feature amount is read in accordance with the value of the flag information. Specifically, when the value of the flag information indicates that the product characteristic information is attached, the feature amount is read from an area other than the position where the characteristic symbol CS is present, and the product characteristic information is not attached. In the case of showing, the feature amount is read from an arbitrary position in the image.

  Moreover, in the said embodiment, since the imaging condition of the goods G was brought close to the imaging condition at the time of imaging of a 2nd product image, the form imaged on the basis of a characteristic symbol was demonstrated, but it is not restricted to this, For example, fruit By capturing the second product image and the product G on the basis of, for example, the vegetable nape portion, etc., both imaging conditions may be made substantially equal without using characteristic symbols.

  In the above embodiment, the product code and the product characteristic code are transmitted from the product reading device 101 to the POS terminal 11. However, the present invention is not limited to this, and product information and products corresponding to each of the product code and the product characteristic code are used. The characteristic information may be transmitted to the POS terminal 11.

  In the above embodiment, the POS terminal 11 includes the PLU file F1 and the product characteristic file F2. However, the present invention is not limited thereto, and the product reading apparatus 101 may include the PLU file F1 and the product characteristic file F2. The external device accessible by the POS terminal 11 and the product reading device 101 may include the PLU file F1 and the product characteristic file F2.

  In the above embodiment, the POS terminal 11 and the commodity reading apparatus 101 are configured as two units. However, the present invention is not limited to this, and a single-unit apparatus having both functions of the POS terminal 11 and the commodity reading apparatus 101 may be used.

  In the above embodiment, the product information and the product characteristic information are output (display output and print output) in the POS terminal 11. However, the present invention is not limited to this, and the product identification unit 1612 recognizes the product reader 101. The product information and the product characteristic information corresponding to the product code of the product and the product characteristic code specified by the individual identification unit 1613 may be displayed on the display device 106 or the customer display device 109.

  In addition, the program executed by each device of the above embodiment is provided by being incorporated in advance in a storage medium (ROM or storage unit) included in each device, but is not limited thereto, and can be installed in a form or executable. Various types of files may be recorded and provided on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). Furthermore, the storage medium is not limited to a medium independent of a computer or an embedded system, but also includes a storage medium that downloads and stores or temporarily stores a program transmitted via a LAN, the Internet, or the like.

  Further, the program executed by each device of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network, or via a network such as the Internet. It may be configured to be provided or distributed.

1 Checkout system 11 POS terminal 61 CPU
611 Information Reception Unit 612 Product Information Acquisition Unit 613 Product Characteristic Information Acquisition Unit 614 Sales Registration Unit 615 Accounting Processing Unit 101 Product Reading Device 105 Touch Panel 106 Display Device 107 Keyboard 161 CPU
164 Imaging unit 165 Audio output unit 1611 Image capture unit 1612 Product identification unit 1613 Individual identification unit 1614 Product information display unit 1615 Information transmission unit F1 PLU file F2 Product characteristic file

JP 2009-217847 A

Claims (5)

A product code indicating the product type, a reference image captured for each product belonging to the product code type so as to include a symbol attached to the product, and characteristic information specific to the individual imaged in the reference image And a first storage means for storing
A feature amount of a captured image obtained by imaging a product to be identified is compared with a feature amount of a region other than the region where the symbol included in the reference image of each product stored in the first storage unit is present. , Product type identifying means for identifying the product code of the product to be identified based on the similarity between both feature quantities;
Detecting means for detecting the symbol attached to the product from a captured image of the product whose product code is identified by the product type identifying unit;
When the symbol is detected by the detection means, a feature amount of an area where the symbol included in each of the reference images stored in association with the product code identified by the product identification means, An individual identifying means for comparing a feature amount of a captured image and identifying a reference image of an individual corresponding to the product to be identified based on the similarity between both feature amounts;
Output means for outputting the characteristic information stored in association with the reference image identified by the individual identification means ;
An information processing apparatus comprising: A second storage means for storing flag information indicating the presence or absence of the characteristic information in association with each product code;
The product identification means stores a product code associated with flag information indicating the presence of the characteristic information among the product codes stored in the first storage means in association with the product code. The product code associated with flag information indicating the absence of the characteristic information is compared using the feature quantity of the other region of the reference image, and any of the reference images stored in association with the product code is stored. Comparison using feature values of
When the product code associated with flag information indicating the absence of the characteristic information is identified by the product identification unit as the product code of the product to be identified, the output unit outputs information on the product code type The information processing apparatus according to claim 1.   The information processing apparatus according to claim 1, further comprising an imaging condition instructing unit that instructs to change an imaging condition related to imaging of the product to be identified when the symbol is not detected by the detecting unit.   The information processing apparatus according to claim 1, wherein the reference image is an image captured under the same imaging condition with the symbol as a center. The computer of the information processing device
A product code indicating the product type, a reference image captured for each product belonging to the product code type so as to include a symbol attached to the product, and characteristic information specific to the individual imaged in the reference image And a first storage means for storing
A feature amount of a captured image obtained by imaging a product to be identified is compared with a feature amount of a region other than the region where the symbol included in the reference image of each product stored in the first storage unit is present. , Product type identifying means for identifying the product code of the product to be identified based on the similarity between both feature quantities;
Detecting means for detecting the symbol attached to the product from a captured image of the product whose product code is identified by the product type identifying unit;
When the symbol is detected by the detection means, a feature amount of an area where the symbol included in each of the reference images stored in association with the product code identified by the product identification means, An individual identifying means for comparing a feature amount of a captured image and identifying a reference image of an individual corresponding to the product to be identified based on the similarity between both feature amounts;
Output means for outputting the characteristic information stored in association with the reference image identified by the individual identification means ;
Program to make it function.

Images