JP5496971B2 - Reader, reader system, and program - Google Patents

Description

  Embodiments described herein relate generally to a reading device, a reading system, and a program.

  There is a code reader that optically captures an image such as a data code attached to a product using an image sensor device and reads information such as a data code from the captured image. Such a code reader stores information reading results from captured images in a storage medium. Thereby, the occurrence of misreading of information is detected by referring to the reading result stored in the storage medium.

JP 2009-181499 A

  By the way, in the conventional code reader, when information misreading occurs, in order to reproduce the information misreading, the information misreading is performed by repeatedly capturing the image and reading the information from the captured image. Analysis of the cause of the occurrence. However, in the conventional code reading device, when analyzing the cause of the misreading of information, the information is read from a newly captured image. When the image is taken, there is a problem that misreading of information is not reproduced, and the situation where information is actually misread is not known.

The reader of the embodiment recognizes a product as a specific object by recognizing a surface state as a feature amount from an imaging unit that captures an image at a preset frame rate, and the recognized image. A reading unit that reads out product information associated with a product and a storage unit that stores the captured image in the storage unit, and the reading unit out of the captured images has a product as a specific object. Storage control means for associating only the recognized image with the product information read from the image and storing it in the storage unit.

The perspective view which shows a checkout system. The block diagram which shows the hardware constitutions of a POS terminal and a goods reading apparatus. The conceptual diagram which illustrates the data structure of a PLU file. The block diagram which shows the function structure of a POS terminal and a goods reading apparatus. The figure for demonstrating the memory | storage operation | movement of the information read from the frame image and the said frame image. The flowchart which shows an example of operation | movement of the checkout system concerning this embodiment.

  Hereinafter, a reading apparatus, a reading system, and a program according to the present embodiment will be described with reference to the drawings, taking a checkout system as an example. The reading system is a checkout system (POS system) including a POS (Point Of Sales) terminal that performs registration and settlement of commodities related to one transaction. This embodiment is an application example to a checkout system introduced in a store such as a supermarket.

  FIG. 1 is a perspective view showing a checkout system. As shown in FIG. 1, the checkout system 1 includes a POS terminal 11 that performs registration and settlement of commodities related to one transaction. In the checkout system 1, the POS terminal 11 is placed on the upper surface of the drawer 21 on the checkout table 51. The POS terminal 11 controls the opening operation of the drawer 21. The POS terminal 11 has a keyboard 22 that is pressed down by an operator (user) on the top surface. Further, the POS terminal 11 is provided with a display device 23 that displays information toward the operator on the back side of the keyboard 22 when viewed from the operator who operates the keyboard 22. The display device 23 displays information on the display surface 23a. In the display device 23, a touch panel 26 is laminated on the display surface 23a. The POS terminal 11 has a customer display device 24 erected on the far side of the display device 23 so as to be rotatable. The customer display device 24 displays information on the display surface 24a. The customer display device 24 shown in FIG. 1 has the display surface 24a facing the front side in FIG. 1, but the customer display device 24 is rotated so that the display surface 24a faces the back side in FIG. Thus, the customer display device 24 displays information to the customer.

  The checkout system 1 is provided with a horizontally long counter-shaped counter table 151 so as to form an L shape with the checkout table 51 on which the POS terminal 11 is placed. The counter table 151 has a load receiving surface 152 on which a shopping basket 153 for storing the product A is placed on the upper surface. The shopping basket 153 can be divided into a first shopping basket 153a brought in by a customer and a second shopping basket 153b positioned at a position sandwiching the commodity reading device 101 from the first shopping basket 153a.

  In the checkout system 1, the commodity reading device 101 connected to the POS terminal 11 so as to be able to transmit and receive data is installed on the receiving surface 152 of the counter stand 151. The commodity reading apparatus 101 includes a thin rectangular housing 102. In the commodity reading apparatus 101, a reading window 103 is arranged in front of the housing 102. In addition, the commodity reading apparatus 101 has a display / operation unit 104 attached to an upper portion of the housing 102. The display / operation unit 104 includes a display device 106 on which a touch panel 105 is stacked. A keyboard 107 is disposed on the right side of the display device 106. The display / operation unit 104 has a card reading groove 108 of a card reader (not shown) on the right side of the keyboard 107. Further, the display / operation unit 104 has a customer display device 109 for providing information to the customer on the back left side of the display / operation unit 104 when viewed from the operator.

  Such a product reading apparatus 101 includes a product reading unit 110 (see FIG. 2). The merchandise reading unit 110 has an imaging unit 164 (see FIG. 2) disposed behind the reading window 103.

  The first shopping basket 153a brought in by the customer stores the commodity A related to one transaction. The operator moves the product A stored in the first shopping basket 153a to the second shopping basket 153b. Then, the operator directs the product A toward the reading window 103 of the product reading device 101 in the process of moving from the first shopping cart 153a to the second shopping cart 153b. At this time, the imaging unit 164 (see FIG. 2) arranged on the back side of the reading window 103 images the product A. The merchandise reading apparatus 101 outputs an image captured by the imaging unit 164 to the POS terminal 11. The POS terminal 11 reads information such as a product ID specifying the product A from the image output from the product reading device 101. The POS terminal 11 refers to the PLU file F1 (see FIG. 3), and stores information related to sales registration, such as the product classification, product name, and unit price of the product A specified by the read product ID. Record sales (not shown) and register sales.

  FIG. 2 is a block diagram illustrating a hardware configuration of the POS terminal and the commodity reading apparatus. The POS terminal 11 includes a microcomputer 60 as an information processing unit that executes information processing. In the microcomputer 60, a ROM (Read Only Memory) 62 and a RAM (Random Access Memory) 63 are connected by a bus to a CPU (Central Processing Unit) 61 that executes various arithmetic processes and controls each unit.

  The POS terminal 11 is connected to the CPU 61 by connecting the above-described drawer 21, keyboard 22, display device 23, touch panel 26, and customer display device 24 via various input / output circuits (all not shown). Yes. The CPU 61 controls the drawer 21, the keyboard 22, the display device 23, the touch panel 26, and the customer display device 24.

  The keyboard 22 includes a numeric keypad 22d, a temporary tightening key 22e, and a tightening key 22f on which numbers such as “1”, “2”, “3”.

  The POS terminal 11 has an HDD (Hard Disk Drive) 64 connected to the CPU 61. The HDD 64 stores programs and various files. When the POS terminal 11 is activated, the CPU 61 copies all or a part of the programs and various files stored in the HDD 64 to the RAM 63 and sequentially executes them. An example of a program stored in the HDD 64 is a product sales data processing program PR. An example of a file stored in the HDD 64 is a PLU file F1 distributed and stored from the store computer SC.

  The PLU file F1 is a file in which an association between information related to sales registration of the product A and an image of the product A is set for each of the products A displayed and sold in the store. FIG. 3 is a conceptual diagram illustrating the data structure of a PLU file. As shown in FIG. 3, the PLU file F <b> 1 includes, for each product A, a product ID uniquely assigned, a product classification to which the product A belongs, a product name, a unit price, information related to sales registration of the product A, and a product A Is a file storing product images obtained by capturing images.

  Returning to FIG. 2, the POS terminal 11 is connected to the CPU 61 via the input / output circuit (not shown) a communication interface 25 for executing data communication with the store computer SC. The store computer SC is installed in a store backyard or the like. The HDD (not shown) of the store computer SC stores the PLU file F1 distributed to the POS terminal 11.

  Further, the POS terminal 11 is connected to the CPU 61 by a connection interface 65 for transmitting / receiving data to / from the commodity reading apparatus 101. The product reading apparatus 101 is connected to the connection interface 65. The POS terminal 11 is connected to the CPU 61 with a printer 66 that prints on receipts and the like. The printer 66 prints the transaction content of one transaction on a receipt under the control of the CPU 61.

  The commodity reading apparatus 101 also includes a microcomputer 160. The microcomputer 160 has a ROM 162 and a RAM 163 connected to the CPU 161 via a bus. The ROM 162 stores a program executed by the CPU 161. The commodity reading apparatus 101 connects the imaging unit 164 and the audio output unit 165 to the CPU 161 via various input / output circuits (not shown). The CPU 161 controls operations of the imaging unit 164 and the audio output unit 165. The product reading apparatus 101 connects the display / operation unit 104 to the POS terminal 11 via the connection interface 176. The CPU 61 of the POS terminal 11 controls the operation of the display / operation unit 104.

  The imaging unit 164 is a color CCD (Charge Coupled Device) image sensor, a color CMOS (Complementary Metal Oxide Semiconductor) image sensor, or the like. Under the control of the CPU 161, an image is read from the reading window 103 at a preset frame rate. Take an image. For example, the imaging unit 164 captures a 30 fps moving image. The imaging unit 164 stores the frame images sequentially captured at a preset frame rate in the RAM 163. In the present embodiment, the imaging unit 164 captures a frame image at a preset frame rate regardless of whether the product A is directed to the reading window 103 by the operator. Therefore, the captured frame image includes an image that does not include the image of the product A.

  The audio output unit 165 is an audio circuit and a speaker for generating a preset warning sound or the like. The sound output unit 165 performs notification by sound such as a warning sound under the control of the CPU 161.

  Further, the commodity reading apparatus 101 is connected to the connection interface 175 that connects the CPU 161 to the connection interface 65 of the POS terminal 11 and enables data transmission / reception with the POS terminal 11. The CPU 161 outputs a frame image (an image including all or part of the product A) captured by the imaging unit 164 of the product reading apparatus 101 via the connection interface 175, and outputs it to the POS terminal 11 via the connection interface 65. input.

  Next, functional units of the CPU 161 and the CPU 61 that are realized by the CPU 161 and the CPU 61 sequentially executing programs will be described with reference to FIG. FIG. 4 is a block diagram illustrating functional configurations of the POS terminal and the commodity reading apparatus. As shown in FIG. 4, the CPU 161 has functions as a captured image capturing unit 401 and an image output unit 402 by sequentially executing programs. Similarly, the CPU 61 has functions as a reading unit 403, a storage control unit 404, and a product registration unit 405.

  The captured image capturing unit 401 outputs an imaging on signal to the imaging unit 164 and causes the imaging unit 164 to start an imaging operation. The captured image capturing unit 401 sequentially captures frame images captured by the imaging unit 164 and stored in the RAM 163 after the imaging operation is started. Captured frame images by the captured image capturing unit 401 are performed in the order stored in the RAM 163.

  The image output unit 402 sequentially outputs the frame images captured by the captured image capturing unit 401 to the POS terminal 11 via the connection interface 175.

  The reading unit 403 includes a product ID that identifies the product A from the frame image captured by the captured image capturing unit 401 in the order output by the image output unit 402 (that is, in the order captured by the imaging unit 164). Information such as service information indicating services such as discounts to be given for purchase of the product A is read. In the present embodiment, since a frame image is captured regardless of whether or not the product A is directed to the reading window 103, the reading unit 403 reads information from all the frame images captured by the imaging unit 164. In other words, the reading unit 403 reads information regardless of whether or not the product A is included in the captured frame image.

  Specifically, the reading unit 403 detects an image of a code symbol such as a barcode or a QR code (registered trademark) included in the captured frame image. Next, the reading unit 403 reads the product ID of the product A from the detected code symbol image. For example, when reading a product ID from a barcode included in a frame image, the reading unit 403 first binarizes the frame image captured by the imaging unit 164. Then, the reading unit 403 detects whether or not a data area regarding the barcode exists in the binarized data. When the reading unit 403 detects the data area related to the barcode in the captured frame image, the reading unit 403 reads the barcode data (JAN code) obtained by decoding the data area related to the barcode as the product ID of the product A. The barcode data includes a manufacturer code, a product ID, a check digit, and the like.

  Further, the reading unit 403 performs character recognition such as OCR (Optical Character Reader) on the frame image captured by the imaging unit 164, thereby reducing the discount printed on the discount sticker attached to the product A or the like. A value or the like can also be read as service information.

  In the present embodiment, the reading unit 403 reads the product ID as information specifying the product A from the code symbol included in the frame image captured by the imaging unit 164, but the present invention is not limited to this. . For example, the reading unit 403 refers to the product image of the PLU file F1 stored in the HDD 64, and from the whole or a part of the product A included in the captured frame image, as a feature amount, a color tone or surface unevenness The product A is recognized as a specific object by reading the surface state such as the situation. And the reading part 403 may implement | achieve the process which reads merchandise ID from the taken-in frame image by reading merchandise ID matched with the merchandise image of the recognized merchandise A from PLU file F1. Note that the reading unit 403 does not consider the outline or size of the product A in order to shorten the processing time.

  Recognizing an object included in an image in this way is called generic object recognition. Regarding such general object recognition, various recognition techniques are described in the following documents.

Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>
Further, techniques for performing general object recognition by dividing an image into regions for each object are described in the following documents.

Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [searched August 10, 2010], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>
The storage control unit 404 stores the frame image output by the image output unit 402 (that is, the frame image captured by the imaging unit 164) in the HDD 64, and the reading unit 403 among the captured frame images. The frame image from which the information has been read is stored in the HDD 64 in association with the information read from the frame image.

  In the present embodiment, the storage control unit 404 stores the frame image in which information is read by the reading unit 403 among the frame images captured by the imaging unit 164 in association with the information read from the frame image in the HDD 64. Shall be allowed to. As a result, it is possible to prevent the frame image from which information has not been read from being stored in the HDD 64, so that the amount of frame image stored in the HDD 64 can be reduced.

  In the present embodiment, the storage control unit 404 stores the captured frame image and information read from the frame image in the HDD 64 of the POS terminal 11, but the present invention is not limited to this. 25, the captured frame image and information read from the frame image are transmitted to the store computer SC, which is a higher system of the POS terminal 11, and the captured image is captured in a storage unit such as an HDD provided in the store computer SC. The read frame image and information read from the frame image may be stored. Alternatively, the storage control unit 404 creates a file in which the captured frame image, the time stamp of the frame image, and information read from the frame image are associated once a day, and the created file is stored in the store computer SC. It may be transmitted and stored in the storage unit of the store computer SC.

  The product registration unit 405 includes information related to sales registration of the product A identified by the product ID read by the reading unit 403, that is, the product classification associated with the product ID read by the reading unit 403 in the PLU file F1. Information related to the sales registration of the product A such as the product name and unit price is recorded in the sales master file and registered.

  FIG. 5 is a diagram for explaining a storage operation of a frame image and information read from the frame image. When the seven images 01 to 07 (frame images) are captured in the order of the images 01 to 07, the reading unit 403 performs decoding of the JAN code from the images 01 to 07 according to the captured order. When the JAN code is decoded from the barcode included in the image 02, the storage control unit 404 causes the HDD 64 to store the image 02 and the JAN code decoded from the barcode included in the image 02 in association with each other. .

  Further, a part of the information of the JAN code is read from the bar code attached to the product A from the image 04, and the remaining information of the JAN code is read from the bar code attached to the product A from the image 05. When the JAN code is decoded based on the read information and the information read from the image 05, the storage control unit 404 reads the image 04, the image 05, the information read from the image 04, and the information read from the image 05. Are stored in the HDD 64 in association with each other.

  Further, the storage control unit 404 discards the images 01, 03, 06, and 07 in which the JAN code has not been read out of the images 01 to 07 without storing them in the HDD 64. This eliminates the need to store in the HDD 64 frame images that are not analyzed in the event of misreading, that is, images whose information has not been read by the reading unit 403. Therefore, the amount of frame images stored in the HDD 64 is reduced. Can be reduced. In the storage operation shown in FIG. 5, the case where the JAN code is decoded based on the information read from the two images of the image 04 and the image 05 is described. However, the JAN code is decoded based on the two images. The present invention is not limited to the case, and the present invention can also be applied to a case where a JAN code is decoded based on information read from two or more images. In this case, the storage control unit 404 associates the two or more images used for decoding the JAN code with the JAN code decoded based on the information read from the two or more images, and stores them in the HDD 64. To do.

  Next, the operation flow of the checkout system 1 will be described. FIG. 6 is a flowchart showing an example of the operation of the checkout system according to the present embodiment.

  First, the operation on the commodity reading apparatus 101 side will be described. As illustrated in FIG. 6, when processing is started in response to the start of product registration by the POS terminal 11, the captured image capturing unit 401 outputs an imaging on signal to the imaging unit 164 and performs imaging by the imaging unit 164. Is started (Act 601). Next, the captured image capturing unit 401 captures the frame image captured by the imaging unit 164 and stored in the RAM 163 (Act 602). Next, the image output unit 402 sequentially outputs the frame images captured by the captured image capture unit 401 to the POS terminal 11 in the captured order (Act 603).

  Next, the CPU 161 determines the presence / absence of the end of the business from the POS terminal 11 based on the end notification of the product registration (Act 604). When the business is continued (No in Act 604), the CPU 161 returns the process to Act 602 and continues the process. When the work is to be ended (Yes in Act 604), the captured image capturing unit 401 outputs an imaging off signal to the imaging unit 164, ends the imaging by the imaging unit 164 (Act 605), and ends the process.

  Next, the operation on the POS terminal 11 side will be described. As shown in FIG. 6, when processing is started in response to the start of product registration in response to an operation instruction on the keyboard 22, the CPU 61 receives the frame image output from the product reading device 101 (Act 606). Next, the reading unit 403 reads information such as a product ID and service information from the frame image captured by the captured image capturing unit 401 in the order output from the image output unit 402 (Act 607).

  Then, the storage control unit 404 associates the frame image whose information is read by the reading unit 403 among the frame images captured by the captured image capturing unit 401 with the information read from the frame image in the HDD 64. Store (Act 608).

  Next, the product registration unit 405 refers to the PLU file F1 stored in the HDD 64, reads information related to the sales registration associated with the product ID read by the reading unit 403, and reads the information related to the read sales registration. Is recorded in the sales master file, and sales of the product A is registered (Act 609). Note that when the service information is read from the frame image by the reading unit 403, the product registration unit 405 adds the service information to the information related to sales registration and records the information in the sales master file.

  Next, the CPU 61 determines whether or not there is an end of business due to the end of sales registration by an operation instruction of the keyboard 22 (Act 610). When continuing the business (No in Act 610), the CPU 61 returns the process to Act 606 and continues the process. When the business is to be ended (Yes in Act 610), the CPU 61 ends the process.

  As described above, according to the checkout system 1 according to the present embodiment, the imaging unit 164 that captures a frame image, the reading unit 403 that reads information from the captured frame image, and the captured frame image are stored in the HDD 64. A storage control unit 404 that stores the information read from the frame image in the HDD 64 in association with the frame image from which the information has been read out of the captured frame image. When analyzing the cause of misreading, the cause of the misreading can be analyzed using the frame image stored in association with the misreading information when analyzing the cause of the misreading. Becomes easier.

  Programs executed by the POS terminal 11 and the commodity reading apparatus 101 according to the present embodiment are provided by being incorporated in advance in a ROM or the like. The programs executed by the POS terminal 11 and the commodity reading apparatus 101 of the present embodiment are files in an installable format or executable format, and are CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). ) Or the like may be recorded and provided on a computer-readable recording medium.

  Further, the program executed by the POS terminal 11 and the commodity reading apparatus 101 according to the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. good. Further, the program executed by the POS terminal 11 and the commodity reading apparatus 101 according to the present embodiment may be configured to be provided or distributed via a network such as the Internet.

  The program executed by the POS terminal 11 according to the present embodiment has a module configuration including the above-described units (reading unit 403, storage control unit 404, product registration unit 405, etc.), and CPU ( The processor) reads out the program from the ROM and executes it, whereby the above-described units are loaded onto the main storage device, and the reading unit 403, the storage control unit 404, and the product registration unit 405 are generated on the main storage device. ing.

  The program executed by the commodity reading apparatus 101 according to the present embodiment has a module configuration including the above-described units (the captured image capturing unit 401 and the image output unit 402). As actual hardware, a CPU (processor) is used. When the program is read from the ROM and executed, the above-described units are loaded on the main storage device, and the captured image capturing unit 401 and the image output unit 402 are generated on the main storage device.

  In the present embodiment, the reading unit 403, the storage control unit 404, and the product registration unit 405 are provided on the POS terminal 11 side. However, the present invention is not limited to this, and the reading unit 403, A storage control unit 404 and a product registration unit 405 may be provided.

  Further, in the present embodiment, the captured image capturing unit 401 and the image output unit 402 are provided on the product reading apparatus 101 side, but the present invention is not limited thereto, and the captured image capturing unit 401 is disposed on the POS terminal 11 side. The image output unit 402 may be provided.

  Although the embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. This embodiment is included in the scope and gist of the invention, and is also included in the invention described in the claims and the equivalent scope thereof.

1 Checkout system 11 POS terminal 64 HDD
101 Product Reading Device 164 Imaging Unit 403 Reading Unit 404 Storage Control Unit

Claims (3)

An imaging unit that captures an image at a preset frame rate;
Reading means for recognizing a product as a specific object by reading a surface state as a feature amount from the captured image, and reading product information associated with the recognized product;
The captured image is stored in a storage unit, and only the image of the captured image recognized as a specific object by the reading unit is read from the image. Storage control means for associating and storing the product information in the storage unit;
A reading apparatus comprising: An imaging unit that captures an image at a preset frame rate;
Reading means for recognizing a product as a specific object by reading a surface state as a feature amount from the captured image, and reading product information associated with the recognized product;
A storage unit for storing the captured image;
A storage control unit that stores the product information read from the image in the storage unit in association with only the image in which the product is recognized as a specific object by the reading unit among the captured images. When,
Reading system. Computer
An image captured at a preset frame rate is stored in a storage unit, and a product is recognized as a specific object by reading a surface state as a feature amount from the captured image. Reading means for reading product information associated with the product;
A storage control unit that stores the product information read from the image in the storage unit in association with only the image in which the product is recognized as a specific object by the reading unit among the captured images. ,
Program to function as.

Images