JP6708775B2 - Product reading device, store accounting system and program - Google Patents

Description

Embodiments of the present invention relate to a store accounting system, a product reading device using an image capturing unit, and a program for causing a computer to function as the product reading device.

Conventionally, a store system has been proposed in which the feature amount of each product is extracted from image data obtained by capturing an image of a product group and compared with a feature amount for collation prepared in advance to recognize the type of each product. ..

JP, 2013-175890, A

In the prior art, in order to register the sales of the next product group, it was necessary to manually give an opportunity to start recognition after removing the products already registered for sales. One of the challenges is to reduce the time loss from the situation in which a product already registered for sale can be removed and the next product can be recognized until the next recognition is started because the device is manually triggered.

In one embodiment, a store accounting system includes an image capturing unit that captures an image of a container containing products, a first image captured by the image capturing unit, and a second image captured after the first image. , A third image captured after the second image and a storage unit that stores the third image, and the first image and the second image stored in the storage unit have different portions from each other. In the case where a plurality of products can be identified in the above, each of the identified plurality of products is registered for sale, and one of the registered products for sale that can be identified on the second image on the third image cannot be identified. In addition, a processing unit for registering sales of a newly identified product different from the products for which sales can be identified on the second image, and an accounting processing unit for accounting for the price of the registered sales product. Prepare

The external view of the store accounting system of embodiment. FIG. 3 is a block diagram showing a hardware configuration of a product reading device and a POS terminal according to the embodiment. The schematic diagram which shows the data structure of the object data file of embodiment. The schematic diagram which shows the data structure of the sales registration list data of embodiment. 3 is a flowchart showing the processing of the CPU of the product reading device of the embodiment. 3 is a flowchart showing processing of the CPU of the POS terminal of the embodiment. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket from which all the goods of an embodiment were taken out. 3 is a flowchart showing the processing of the CPU of the product reading device of the embodiment. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket from which all the goods of an embodiment were taken out. 3 is a flowchart showing the processing of the CPU of the product reading device of the embodiment. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods of an embodiment were put. The figure showing the basket in which the goods were displayed which the panel display part of an embodiment displays. The figure showing the basket from which all the goods of an embodiment were taken out. The external view of the store accounting system of embodiment. FIG. 3 is a block diagram showing a hardware configuration of a product reading device and a POS terminal according to the embodiment. 3 is a flowchart showing the processing of the CPU of the product reading device of the embodiment. The figure showing the goods list which the clear display of an embodiment displays.

Hereinafter, an embodiment of a product reading device will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an external view of a store accounting system according to an embodiment. This system includes a commodity reading device 1 and a POS (Point Of Sales) terminal 2 as an accounting processing unit.

The commodity reading device 1 is mounted on the checkout counter 3. The POS terminal 2 is installed on the cash register 4 via a drawer 5. The commodity reading device 1 and the POS terminal 2 are electrically connected by a communication cable. The checkout counter 3 has an elongated shape along the customer passage on the far side. The cashier table 4 is placed substantially perpendicular to the checkout counter 3 on the downstream side of the end of the checkout counter 3 in the moving direction of the customer who moves along the checkout counter 3. Then, the front side of the checkout counter 3 and the front side of the cashier table 4 are spaces for the clerk.

The merchandise reading device 1 is a plate-shaped body portion 10A that is erected at a substantially central portion of an edge of the checkout counter 3 on the customer passage side, and a front upper end portion of the body portion 10A, and is substantially orthogonal to the front surface. The eaves portion 10B protruding in the direction (the direction facing the checkout counter 3) is provided. Then, a keyboard 11, a touch panel 12, and a customer display 13 are attached to the top of the body portion 10A as devices related to product registration. Further, the imaging unit 14 is built in the eaves unit 10B.

The image pickup unit 14 includes a CCD (Charge Coupled Device) image pickup element which is an area image sensor and a drive circuit thereof, and an image pickup lens for forming an image of an image pickup area on the CCD image pickup element. The imaging area refers to an area of an image formed on the area of the CCD imaging device through the imaging lens from the reading window formed on the surface of the eaves portion 10B facing the checkout counter 3. The image pickup unit 14 outputs an image of the image pickup area formed on the CCD image pickup element through the image pickup lens.

The accounting counter 3 is provided with a basket 6 as a container in which the products purchased by the customer are placed. The store clerk puts the basket 6 containing the product under the eaves portion 10B of the checkout counter 3. The image of the basket 6 containing the product is captured by the image capturing unit 14, and the captured image is displayed on the touch panel 12 by the function of the product reading device 1. The image capturing area of the image capturing unit 14 is below the eaves portion 10b of the checkout counter 3.

The POS terminal 2 includes a keyboard 21, an operator display 22, a customer display 23, and a receipt printer 24 as devices for settlement of commercial transactions.

FIG. 2 is a block diagram showing a hardware configuration of the commodity reading device 1 and the POS terminal 2.

The commodity reading apparatus 1 has a CPU (Central Processing Unit) 101 as a processing unit. A ROM 103 and a RAM 104 are connected to the CPU 101 via a bus line 102 such as an address bus and a data bus. The ROM 103 stores the program executed by the CPU 101. The RAM 104 stores data of the object data file 7 transmitted from the POS terminal 2 and data of a sales registration list 8 as a list of products registered for sales. The image pickup unit 14 is connected to the bus line 102. The keyboard 11, the touch panel 12, and the customer display 13 are connected to the bus line 102 via the connection interface 105 and the connection interface 106. The touch panel 12 includes, for example, a panel display unit 121 using a liquid crystal display, and a touch panel sensor 122 arranged on the screen of the panel display unit 121 in an overlapping manner.

The POS terminal 2 is equipped with a CPU 201. A ROM 203, a RAM 204, an HDD (Hard Disk Drive) device 205, a communication interface 206, and a connection interface 207 are connected to the CPU 201 via a bus line 202. A keyboard 21, an operator display 22, a customer display 23, a printer 24 and a drawer 5 are connected to the bus line 202. The communication interface 206 is a network adapter such as a LAN (Local Area Network). The POS terminal 2 receives the object data file 7 from the store computer SC. The HDD 205 stores the received object data file 7. The connection interface 207 connects to the connection interface 105 and the connection interface 106 of the commodity reading apparatus 1 via a communication cable. The POS terminal 2 transmits/receives data to/from the commodity reading device 1 via the connection interface 207. The POS terminal 2 receives the product information read by the product reading device 1. The POS terminal 2 sends the data in the object data file 7 to the product reading device 1. A program executed by the CPU 201 is stored in the ROM 203. The name and unit price of the product are stored in the RAM 204.

FIG. 3 is a schematic diagram showing the data structure of the object data file 7. The object data file 7 stores the name, unit price, appearance feature parameter, etc. of the product in association with the product code for identifying each product. The appearance feature parameter is a standard appearance feature amount of the product such as the shape of the product, the hue of the surface, the pattern, the unevenness, etc. encoded for each product. In the object data file 7, appearance feature parameters of the basket 6 and the like are stored in association with a code indicating the end of sales registration.

FIG. 4 is a schematic diagram showing the data structure of the sales registration list data 8. The sales registration list data 8 stores the product code of each registered product. The name, unit price, position/orientation information, etc. of the product may be stored in association with the product code. The position and orientation information is obtained by encoding the position and orientation of the product in the imaging area.

FIG. 5 is a flowchart showing the processing of the CPU 101 of the commodity reading device 1.

The CPU 101 starts processing in response to a signal generated by a store clerk who sets the basket 6 in which the product is placed in the image capturing area below the image capturing unit 14 of the transaction counter 3 and operates the keyboard 11.

The CPU 101 sets the processing start flag to 0 (501). The CPU 101 captures an image of the image capturing area with the image capturing unit 14 (502). The CPU 101 saves the image of the imaging area in the RAM 104.

An image of an imaging area as a result of the previous imaging is referred to as a previous image G1, and an image of an imaging area as a result of this time imaging is referred to as a current image G2.

When the processing start flag is 0 (No in 503), the CPU 101 does not perform 504 but does 505.

When the process start flag is 1 (Yes in 503), the CPU 101 searches the current image G2 in the RAM 104 for a portion different from the previous image G1 in the RAM 104 (504).

If there is no different part on the current image G2 from the previous image G1 (No in 504), the CPU 101 executes 502 again.

If there is a portion on the current image G2 different from the previous image G1 (Yes in 504), 505 is performed.

The CPU 101 searches the RAM 104 for a portion on the current image G2 that matches the appearance feature parameter of each product defined in the object data file 7 (505).

A portion of the current image G2 on the RAM 104 that matches the appearance feature parameter of the product is called a matching portion.

When no matching portion is found (No in 505), the CPU 101 performs 502 again when the processing start flag is 1 (Yes in 513). If the processing start flag is 0 (No in 513), the processing ends.

When the matching portion is found (Yes in 505), the CPU 101 determines whether or not the matching portion matches the basket 6 (506).

If the matching portion is the portion that matches the basket 6 (Yes in 506), if the processing start flag is 0 (No in 507), the processing ends. If the processing start flag is 1 (Yes in 507), the CPU 101 outputs the sales registration list to the POS terminal 2 via the connection interface 105 (508), and ends the processing.

When the matched portion is not the portion matched with the basket 6 (No in 506), the CPU 101 sets the processing start flag to 1 (509). In the configuration of this flowchart, the processing start flag indicates whether or not the CPU 101 has started the processing with the product in the imaging area.

The CPU 101 saves the mark image in which the matching portion of one product is marked on the RAM 104 (510). The CPU 101 displays an image in which the print image is overlaid on the image G2 this time, on the panel display unit 121, using a region other than the print as a transparent color (511). The CPU 101 registers the product with the matched portion marked in the sales registration list (512). The CPU 101 executes 502 again.

FIG. 6 is a flowchart showing the processing of the CPU 201 of the POS terminal 2. When the sales registration list data is received from the commodity reading device 1 via the connection interface 207, the CPU 201 starts processing. The CPU 201 stores the sales registration list data in the RAM 204 (601). The CPU 201 stores in the RAM 204 the total unit price of the product group stored in the RAM 204 (602). The CPU 201 displays the total unit price of the product group on the operator display 22 and the customer display 23 (603) and waits for the deposit to input the amount of money received from the customer, such as sales registration list data. The state becomes for accounting processing of the price of the above product, and the processing ends.

The operation of the commodity reading device 1 will be described with reference to FIGS. 7 to 17.

FIG. 7 is a diagram showing the basket 6 in which the products are put. The basket 6 contains products such as tissue 61, potato chips 62, udon 63, cheese 64, and chopsticks 65. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64, the udon 63, and the tissue 61, so that the image pickup unit 14 cannot confirm the appearance sufficient for recognition. The cheese 64 is hidden by the tissue 61 and is in a state in which it cannot be seen from the image capturing unit 14 enough for recognition. The udon 63 is hidden by the potato chips 62 and is in a state in which it is not possible to confirm the appearance sufficient for recognition from the imaging unit 14. The tissue 61 and the potato chips 62 are in a state in which they can be seen from the image capturing unit 14 sufficiently for recognition.

FIG. 8 is a diagram showing the basket 6 in which the product is put, which is displayed by the panel display unit 121. When the processing started by the CPU 101 proceeds to 512 when the basket 6 in the state of FIG. 7 is in the imaging area, the panel display unit 121 presents the printed image in which the recognized portion of the recognized tissue 61 is marked. The image overlaid on the image G2 is displayed. The mark is a dotted line that surrounds the matching portion of the product, and the letters "OK" in the range surrounded by the dotted line. At this time, the product code of the recognized tissue is registered in the sales registration list on the RAM 104.

It should be noted that in the state of FIG. 7, both the tissue 61 and the potato chips 62 are in a state in which they can be confirmed from the imaging unit 14 enough for recognition, and thus the potato chips 62 may be recognized first. .. Which is recognized first depends on the configuration, such as from which position of the image in the imaged area the matching portion is searched for, and the order of products for which the matching portion is searched. For example, if you configure the front side of the image in the imaging area to find the matching part toward the customer passage side, the front side product may be registered for sales first, and the front side product may be taken out first. it can. In this way, it is easy to take out the products for which sales have been registered without being disturbed by the products for which sales have not been registered. For example, in the case of sequentially transferring the registered products for sale to the basket 1006 placed on the left of the basket 6, if the left side of the image in the imaging area is searched for the matching part first, the left side product is Sales can be registered first, and the products on the left side can be moved to the basket 1006 first. By doing so, it is easy to move the sales-registered products from the basket 6 to the basket 1006 without being disturbed by the unregistered products. For example, if the order of the products to be searched for the matching portion is configured such that the heavy items and the large items are searched first, it is easy to orderly move the products to the basket 1006.

FIG. 9 is a diagram showing the basket 6 from which the tissue 61 is taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and the udon 63, and a state sufficient for recognition cannot be confirmed from the imaging unit 14. The udon 63 is hidden by the potato chips 62 and is in a state in which it is not possible to confirm the appearance sufficient for recognition from the imaging unit 14. The potato chips 62 and the cheese 64 are in a state in which they can be seen from the image capturing unit 14 sufficiently for recognition.

FIG. 10 is a diagram showing the basket 6 in which the product is displayed, which is displayed by the panel display unit 121 when the tissue 61 is taken out from the basket 6 and the processing further proceeds to 512 after the state of FIG. .. The panel display unit 121 displays an image in which the mark image in which the recognized portion of the potato chips 62 that has been recognized is marked is overlaid on the image G2 this time. At this time, the product code of the newly recognized potato chips is registered in the sales registration list on the RAM 104.

FIG. 11 is a diagram showing the basket 6 from which the potato chips 62 are taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and the udon 63, and a state sufficient for recognition cannot be confirmed from the imaging unit 14. The udon 63 and the cheese 64 are in a state in which the imaging unit 14 can confirm the appearances sufficient for recognition.

FIG. 12 is a view showing the basket 6 in which the product is displayed, which is displayed by the panel display unit 121 when the potato chips 62 are taken out of the basket 6 and the processing further proceeds to 512 after the state of FIG. .. The panel display unit 121 displays an image in which the mark image in which the recognized portion of the udon 63 that has been recognized is marked is overlaid on the image G2 this time. At this time, the product code of the newly recognized udon is registered in the sales registration list on the RAM 104.

FIG. 13 is a diagram showing the basket 6 from which the udon 63 has been taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and cannot be seen from the image capturing unit 14 sufficiently for recognition. The cheese 64 is in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition.

FIG. 14 is a diagram showing the basket 6 in which products are displayed, which is displayed by the panel display unit 121 when the udon 63 is taken out of the basket 6 and the processing further proceeds to 512 after the state of FIG. The panel display unit 121 displays an image in which the mark image in which the matching portion of the recognized cheese 64 is marked is overlaid on the image G2 this time. At this time, the product code of the newly recognized cheese is registered in the sales registration list on the RAM 104.

FIG. 15 is a diagram showing the basket 6 from which the cheese 64 is taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition.

FIG. 16 is a diagram showing the basket 6 in which the product is put, which is displayed by the panel display unit 121 when the cheese 64 is taken out of the basket 6 and the processing further proceeds to 512 after the state of FIG. 14. The panel display unit 121 displays an image in which the mark image in which the matching portion of the recognized cheese 64 is marked is overlaid on the image G2 this time. At this time, the product code of the newly recognized chopsticks is registered in the sales registration list on the RAM 104.

FIG. 17 is a diagram showing the basket 6 from which the chopsticks 65 are taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. All the products have been taken out, and the basket 6 itself set in the image capturing area below the image capturing unit 14 of the transaction counter 3 is in a state in which the image capturing unit 14 can confirm the appearance sufficient for recognition. The chopsticks 65 are taken out of the basket 6 and the process further proceeds to 506. When only the basket 6 is recognized, the sales registration is completed and the sales registration list is output to the POS terminal 2. If the process started by the CPU 101 proceeds to 506 and only the basket 6 is recognized in a state where no products are stored in the basket 6, the sales registration is not started.

The store accounting system according to the first embodiment automatically recognizes a product when a product already registered for sales is removed and the next product can be recognized, so that the time loss until sales registration is reduced. be able to.

Further, the store accounting system according to the first embodiment automatically ends the sales registration when the last product disappears and the basket can be recognized, so that it is possible to reduce the time loss until the accounting is performed after the sales registration is completed. ..

Further, since the store accounting system according to the first embodiment marks the appearance of the products for which sales registration has been performed on the image of the product group, the products that are not registered for sales are erroneously taken out of the basket. It is possible to prevent the product from being stored or packed in a bag.

In the first embodiment, the CPU 101 starts the process in response to a signal generated by operating the keyboard 11, but the process may be started when a matching part of the product is found. With this configuration, the sales registration is automatically started when the first product is recognized, so that the time loss before the sales registration can be reduced.

In the first embodiment, the matching portion is searched for in the current image G2 compared with the previous image G1, but the matching portion is not necessarily searched for in the image to be compared with the previous image G1. For example, one of the images captured at a plurality of time points after the previous image G1 may be used for comparison, and a matching portion may be searched for from the other image.

In the first embodiment, the container 6 is an example of the container, but a container having a box shape, a bag shape, or a container capable of containing a product can be applied as the container.

In the first embodiment, the mark is the dotted line surrounding the matching portion of the product and the character “OK” in the range surrounded by the dotted line, but it may be only the line surrounding the matching portion of the product. The mark can be applied as long as it is a colored surface that surrounds the matching part of the product, or some mark near the matching part of the product, such as a mark that shows which product is the matching part.

In the first embodiment, the sales registration list as the list of sales-registered products includes the product code. However, as described above, the sales registration list includes additional information such as the product name, unit price, and position/orientation information. It may be. The list of products registered for sale can be appropriately applied as long as the POS terminal 2 can recognize the products for accounting processing of the price of the products.

(Modification 1 of the first embodiment)
FIG. 18 is a flowchart showing the processing of the CPU 101 of the commodity reading device 1.

The CPU 101 starts processing in response to a signal generated by a store clerk who sets the basket 6 in which the product is placed in the image capturing area below the image capturing unit 14 of the transaction counter 3 and operates the keyboard 11.

The CPU 101 sets the processing start flag to 0 (1801). The CPU 101 captures an image of the image capturing area with the image capturing unit 14 (1802). The CPU 101 saves the image of the imaging area in the RAM 104.

An image of an imaging area as a result of the previous imaging is referred to as a previous image G1, and an image of an imaging area as a result of this time imaging is referred to as a current image G2.

When the processing start flag is 0 (No in 1803), the CPU 101 performs 1805 without performing 1804.

When the processing start flag is 1 (Yes in 1803), the CPU 101 searches the RAM 104 for a current image G2 that is different from the previous image G1 in the RAM 104 (1804).

If there is no different part from the previous image G1 on the current image G2 (No in 1804), the CPU 101 performs 1802 again.

If there is a different portion on the current image G2 from the previous image G1 (Yes in 1804), 1805 is performed.

The CPU 101 searches the RAM 104 for a portion on the current image G2 that matches the appearance feature parameter of each product defined in the object data file 7 (1805). A portion of the current image G2 on the RAM 104 that matches the appearance feature parameter of the product is called a matching portion. When a plurality of matching parts of a part matching the appearance characteristic parameter of a certain product and a part matching the appearance characteristic parameter of another product are found, they are treated as separate matching parts. If multiple parts that match the appearance feature parameters of one product are found at different positions, treat each as a separate matching part.

If no matching portion is found (No in 1805), the CPU 101 performs 1802 again if the process start flag is 1 (Yes in 1813). If the processing start flag is 0 (No in 1813), the processing ends.

When the matching portion is found (Yes in 1805), the CPU 101 determines whether the matching portion is only the portion matching the basket 6 (1806).

If the matching portion is only the portion that matches the basket 6 (Yes in 1806), the CPU 101 ends the processing if the processing start flag is 0 (No in 1807).

If the process start flag is 1 (Yes in 1807), the CPU 101 outputs the sales registration list to the POS terminal 2 via the connection interface 105 (1808), and ends the process. When the matching portion is not the only portion matching the basket 6 (No in 1806), the CPU 101 sets the processing start flag to 1 (1809). In the configuration of this flowchart, the processing start flag indicates whether or not the CPU 101 has started the processing with the product in the imaging area.

The CPU 101 saves the mark image in which the matching portion of the product is marked on the RAM 104 (1810). When a plurality of matching parts are found in the image G2 this time, the CPU 101 marks each matching part. The CPU 101 saves the mark image with a plurality of marks on the RAM 104.

The CPU 101 displays an image in which the print image is overlaid on the image G2 this time on the panel display unit 121 by using the area other than the print as a transparent color (1811).

The CPU 101 registers, in the sales registration list, a group that has not been registered in the sales registration list among the combinations of the product code and the position/orientation information of the products marked in the matching portion. (1812). The CPU 101 executes 1802 again.

Operations of the commodity reading device 1 and the POS terminal 2 will be described with reference to FIGS. 19 to 29.

FIG. 19 is a diagram showing the basket 6 in which products are put. The basket 6 contains products such as tissue 61, potato chips 62, udon 63, cheese 64, and chopsticks 65. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64, the udon 63, and the tissue 61, so that the image pickup unit 14 cannot confirm the appearance sufficient for recognition. The cheese 64 is hidden by the tissue 61 and is in a state in which it cannot be seen from the image capturing unit 14 enough for recognition. The udon 63 is hidden by the potato chips 62 and is in a state in which it is not possible to confirm the appearance sufficient for recognition from the imaging unit 14. The tissue 61 and the potato chips 62 are in a state in which they can be seen from the image capturing unit 14 sufficiently for recognition.

FIG. 20 is a diagram showing the basket 6 in which products are displayed, which is displayed by the panel display unit 121. When the process started by the CPU 101 proceeds to 1812 when the basket 6 in the state of FIG. 19 is in the imaging area, the panel display unit 121 marks the recognized portion where the tissue 61 and the potato chips 62 match. An image in which the printed image is overlaid on the image G2 this time is displayed. The mark is a dotted line that surrounds the matching portion of the product, and the letters "OK" in the range surrounded by the dotted line. At this time, in the sales registration list on the RAM 104, the name, unit price, and position/orientation information of the product are registered in association with the recognized product code of the tissue. Further, the name, unit price, and position/orientation information of the product are registered in association with the recognized product code of the potato chips.

FIG. 21 is a diagram showing the basket 6 from which the tissue 61 has been taken out of the tissue 61 and the potato chips 62 in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and the udon 63, and a state sufficient for recognition cannot be confirmed from the imaging unit 14. The udon 63 is hidden by the potato chips 62 and is in a state in which the image pickup unit 14 cannot confirm the appearance sufficient for recognition. The potato chips 62 and the cheese 64 are in a state in which they can be seen from the image capturing unit 14 sufficiently for recognition.

22 is a view showing the basket 6 in which the product is displayed, which is displayed by the panel display unit 121 when the tissue 61 is taken out from the basket 6 and the processing further proceeds to 1812 after the state of FIG. .. The panel display unit 121 displays an image in which a mark image in which the recognized portion of the potato chips 62 and the cheese 64 that have been recognized is marked is overlaid on the image G2 this time. At this time, in the sales registration list on the RAM 104, the name, unit price, and position/posture information of the product are registered in association with the product code of the newly recognized cheese. Since the combination of the product code of the potato chips and the position/orientation information has already been recognized and registered in the sales registration list in the state of FIG. 20, the potato chips record is not newly registered.

FIG. 23 is a diagram showing the basket 6 in which the potato chips 62 are taken out of the potato chips 62 and the cheese 64 in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and the udon 63, and a state sufficient for recognition cannot be confirmed from the imaging unit 14. The udon 63 and the cheese 64 are in a state in which the imaging unit 14 can confirm the appearances sufficient for recognition.

FIG. 24 is a view showing the basket 6 in which the product is displayed, which is displayed by the panel display unit 121 when the potato chips 62 are taken out from the basket 6 and the processing further proceeds to 1812 after the state of FIG. . The panel display unit 121 displays an image in which the mark image in which the matching portion between the recognized cheese 64 and the udon 63 is marked is overlaid on the image G2 this time. At this time, in the sales registration list on the RAM 104, the product name, unit price, and position/posture information of the product are registered in association with the newly recognized product code of the udon. Since the combination of the product code of cheese and the position and orientation information has already been recognized and registered in the sales registration list in the state of FIG. 22, the cheese record is not newly registered.

FIG. 25 is a diagram showing the basket 6 from which the udon 63 has been taken out of the cheese 64 and the udon 63 in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and cannot be seen from the image capturing unit 14 sufficiently for recognition. The cheese 64 is in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition.

FIG. 26 is a diagram showing the basket 6 in which the product is displayed, which is displayed by the panel display unit 121 when the udon 63 is taken out of the basket 6 and the processing further proceeds to 1812 after the state of FIG. 24. The panel display unit 121 displays an image in which the mark image in which the matching portion of the recognized cheese 64 is marked is overlaid on the image G2 this time. Since the combination of the product code of cheese and the position and orientation information has already been recognized and registered in the sales registration list in the state of FIG. 24, the cheese record is not newly registered.

FIG. 27 is a diagram showing the basket 6 from which the cheese 64 is taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition.

FIG. 28 is a diagram showing the basket 6 in which the product is displayed, which is displayed by the panel display unit 121 when the cheese 64 is taken out from the basket 6 and the processing further proceeds to 1812 after the state of FIG. 26. The panel display unit 121 displays an image in which the mark image in which the matching portion of the recognized cheese 64 is marked is overlaid on the image G2 this time. At this time, in the sales registration list on the RAM 104, the name, unit price and position/orientation information of the product are registered in association with the newly recognized product code of the chopsticks.

FIG. 29 is a diagram showing the basket 6 from which the chopsticks 65 are taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. All the products have been taken out, and the basket 6 itself set in the imaging area below the imaging unit 14 of the checkout counter 3 is in a state where the imaging unit 14 can confirm a sufficient appearance for recognition. The chopsticks 65 are taken out of the basket 6 and the process further proceeds to 1806. When only the basket 6 is recognized, the sales registration is completed and the sales registration list is output to the POS terminal 2.

The store accounting system according to the first modification automatically recognizes a plurality of products when a product already registered for sales is removed and the next product can be recognized, so that the time loss until sales registration is reduced. be able to. In addition, the options for the order of taking out the products from the basket can be widened, and quick and stable bagging can be facilitated.

(Modification 2 of the first embodiment)
FIG. 30 is a flowchart showing the processing of the CPU 101 of the commodity reading device 1.

The CPU 101 starts processing in response to a signal generated by a store clerk who sets the basket 6 in which the product is placed in the image capturing area below the image capturing unit 14 of the transaction counter 3 and operates the keyboard 11.

The CPU 101 sets the process start flag to 0 (3001). The CPU 101 clears the extraction waiting list on the RAM 104 (3002).

The CPU 101 captures an image of the image capturing area with the image capturing unit 14 (3003). The CPU 101 saves the image of the imaging area in the RAM 104. An image of an imaging area as a result of the previous imaging is referred to as a previous image G1, and an image of an imaging area as a result of this time imaging is referred to as a current image G2.

When the processing start flag is 0 (No in 3004), the CPU 101 executes 3006 instead of 3005.

When the processing start flag is 1 (Yes in 3004), the CPU 101 searches the current image G2 on the RAM 104 for a portion different from the previous image G1 on the RAM 104 (3005). If there is no different part on the current image G2 from the previous image G1 (No in 3005), the CPU 101 performs 3003 again. If there is a different portion from the previous image G1 on the current image G2 (Yes in 3005), 3006 is performed.

The CPU 101 searches the RAM 104 for a portion on the current image G2 that matches the appearance feature parameter of each product defined in the object data file 7 (3006). A portion of the current image G2 on the RAM 104 that matches the appearance feature parameter of the product is called a matching portion. When a plurality of matching parts of a part matching the appearance characteristic parameter of a certain product and a part matching the appearance characteristic parameter of another product are found, they are treated as separate matching parts. If multiple parts that match the appearance feature parameters of one product are found at different positions, treat each as a separate matching part.

When no matching portion is found (No in 3006), the CPU 101 performs 3003 again when the process start flag is 1 (Yes in 3017). If the process start flag is 0 (No in 3017), the process ends.

When the matching portion is found (Yes in 3006), the CPU 101 determines whether the matching portion is only the portion matching the basket 6 (3007).

When the matching portion is only the portion that matches the basket 6 (Yes in 3007), the CPU 101 ends the processing if the processing start flag is 0 (No in 3008). If the processing start flag is 1 (Yes in 3008), the CPU 101 outputs the sales registration list to the POS terminal 2 via the connection interface 105 (3009), and ends the processing.

When the matching portion is not the only portion matching the basket 6 (No in 3007), the CPU 101 sets the processing start flag to 1 (3010). In the configuration of this flowchart, the processing start flag indicates whether or not the CPU 101 has started the processing with the product in the imaging area.

The CPU 101 determines whether or not all the matched products are on the removal waiting list (3011). When the matched product is on the removal waiting list (Yes in 3011), the CPU 101 performs 3003 again. When all the matched products are not in the take-out waiting list (No in 3011), the CPU 101 clears the take-out waiting list (3012).

The CPU 101 saves the mark image in which the matching portion of the product is marked on the RAM 104 (3013). When a plurality of matching parts are found in the image G2 this time, the CPU 101 marks each matching part. The CPU 101 saves the mark image with a plurality of marks on the RAM 104. The CPU 101 displays, on the panel display unit 121, an image in which the print image is overlaid on the image G2 this time by using the area other than the print as a transparent color (3014). The CPU 101 registers the product with the matched portion marked in the sales registration list. (3015).

The CPU 101 registers the merchandise registered in the sales registration list in the removal waiting list. (3016). The CPU 101 executes 3003 again.

Operations of the commodity reading device 1 and the POS terminal 2 will be described with reference to FIGS. 31 to 41.

FIG. 31 is a diagram showing the basket 6 in which products are put. The basket 6 contains products such as tissue 61, potato chips 62, udon 63, cheese 64, and chopsticks 65. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64, the udon 63, and the tissue 61, so that the image pickup unit 14 cannot confirm the appearance sufficient for recognition. The cheese 64 is hidden by the tissue 61 and is in a state in which it cannot be seen from the image capturing unit 14 enough for recognition. The udon 63 is hidden by the potato chips 62 and is in a state in which the image pickup unit 14 cannot confirm the appearance sufficient for recognition. The tissue 61 and the potato chips 62 are in a state in which they can be seen from the image capturing unit 14 sufficiently for recognition.

FIG. 32 is a diagram showing the basket 6 in which products are displayed, which is displayed by the panel display unit 121. When the processing started by the CPU 101 proceeds to 3016 when the basket 6 in the state of FIG. 31 is in the imaging area, the panel display unit 121 marks the recognized portion where the tissue 61 and the potato chips 62 match. An image in which the stamp image is overlaid on the image G2 this time is displayed. The mark is a dotted line that surrounds the matching portion of the product, and the letters "OK" in the range surrounded by the dotted line. At this time, the recognized tissue and potato chips are registered in the sales registration list and the removal waiting list on the RAM 104.

FIG. 33 is a diagram showing the basket 6 in which the tissue 61 is taken out of the tissue 61 and the potato chips 62 in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and the udon 63, and a state sufficient for recognition cannot be confirmed from the imaging unit 14. The udon 63 is hidden by the potato chips 62 and is in a state in which the image pickup unit 14 cannot confirm the appearance sufficient for recognition. The potato chips 62 and the cheese 64 are in a state in which they can be seen from the image capturing unit 14 sufficiently for recognition. When the state of FIG. 32 is reached after the state of FIG. 32 is reached, the potato chips on the removal waiting list are not removed and remain in the basket 6 and can be recognized by the imaging unit 14, so the processing is up to 3011. Even if the user proceeds, the cheese 64 that is in a state in which the image capturing unit 14 is able to confirm the appearance sufficient for recognition is not registered in the sales registration list.

FIG. 34 is a diagram showing the basket 6 from which the potato chips 62 have been taken out after the state of FIG. 33. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and the udon 63, and a state sufficient for recognition cannot be confirmed from the imaging unit 14. The udon 63 and the cheese 64 are in a state in which the imaging unit 14 can confirm the appearances sufficient for recognition. When the state of FIG. 32 is reached after the state of FIG. 32 is reached, the tissues and potato chips on the removal waiting list have been removed from the basket 6 and cannot be recognized by the imaging unit 14, so the processes of 3012 to 3016 are performed. Will be done.

As shown in FIG. 35, the panel display unit 121 displays an image in which the mark image in which the matching portion between the recognized cheese 64 and the udon 63 is marked is overlaid on the image G2 this time. The udon 63 and the cheese 64, which are in a state in which the image capturing unit 14 can confirm the appearance sufficient for recognition, are registered in the sales registration list by the processing of 3015. Further, the udon 63 and cheese 64 registered in the sales registration list are registered in the removal waiting list by the processing of 3016.

FIG. 36 is a diagram showing the basket 6 from which the udon 63 has been taken out of the udon 63 and the cheese 64 in which the matching portion is marked in the image displayed by the panel display unit 121. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are hidden by the cheese 64 and cannot be seen from the image capturing unit 14 sufficiently for recognition. The cheese 64 is in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition. When the state of FIG. 35 is reached after the state of FIG. 35 is reached, the cheese on the take-out waiting list remains in the basket 6 without being taken out and can be recognized by the imaging unit 14, so the process proceeds to 3011. However, the processing of 3012 to 3016 is not performed.

FIG. 37 is a diagram showing the basket 6 from which the cheese 64 has been taken out after the state of FIG. 36. The basket 6 itself set in the image capturing area below the image capturing unit 14 of the checkout counter 3 is hidden by the product, and the image capturing unit 14 cannot confirm the appearance sufficient for recognition. The chopsticks 65 are in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition. When the state of FIG. 37 is reached after the state of FIG. 35 is reached, the udon and cheese on the take-out waiting list are taken out of the basket 6 and cannot be recognized by the imaging unit 14, and therefore the processes of 3012 to 3016 are performed. It

As shown in FIG. 38, the panel display unit 121 displays an image in which the mark image in which the matching portion of the recognized chopstick 65 is marked is overlaid on the image G2 this time. The chopsticks 65 that are in a state in which the image pickup unit 14 can confirm the appearance sufficient for recognition are registered in the sales registration list by the processing of 3015. Further, the chopsticks 65 registered in the sales registration list are registered in the removal waiting list by the processing of 3016.

FIG. 39 is a diagram showing the basket 6 from which the chopsticks 65 have been taken out, in which the matching portion is marked in the image displayed by the panel display unit 121. All the products have been taken out, and the basket 6 itself set in the image capturing area below the image capturing unit 14 of the transaction counter 3 is in a state in which the image capturing unit 14 can confirm the appearance sufficient for recognition. The chopsticks 65 are taken out of the basket 6 and the process further proceeds to 3007. When only the basket 6 is recognized, the sales registration is completed and the sales registration list is output to the POS terminal 2.

The store accounting system according to the second modification automatically recognizes a plurality of products when it becomes possible to recognize the next product by removing all the products already registered for sales, thereby reducing the time loss until sales registration. be able to. In addition, the options for the order of taking out the products from the basket can be widened, and quick and stable bagging can be facilitated.

(Second embodiment)
FIG. 40 is an external view of the store accounting system according to the embodiment. This system includes a commodity reading device 4001 and a POS (Point Of Sales) terminal 2. Since the POS terminal 2 has the same configuration as that of the first embodiment, detailed description will be omitted.

The POS terminal 2 is placed away from the commodity reading device 4001. The POS terminal 2 is placed with the side substantially opposite to the side of the customer or the product reading device 4001 as the front so that the customer operating the product reading device 4001 can easily see from the clerk operating the POS terminal 2. The commodity reading device 4001 and the POS terminal 2 installed on the cash register 4 via the drawer 5 are electrically connected by a communication cable.

The product reading device 4001 includes a body section 4010, a basket scale 4011, a clear display 4012, a support section 4013, an imaging section 4014, a bagging stand 4015, a bag scale 4016, a bag rod 4017, and a hook 4018. An upright body portion 4010 supports a basket scale 4011 and a bag scale 4016 so as to be capable of moving up and down in a front direction thereof (a front direction when viewed from a customer). The customer places the basket 4006 containing the product on the basket scale 4011. The customer places a bag 4019 on which the product is put in the bag scale 4016. The customer can hook the handle of the bag 4019 on the hook 4018 protruding from the bag rod 4017 extending upward from the bag scale 4016 so that the customer can easily put the product in the bag 4019.

The height of the basket scale 4011 and the height of the bag scale 4016 can be adjusted up and down by the customer so that the customers having different heights and postures can easily transfer the products from the basket 4011 to the bag 4019. The support portion 4013 supports the clear display 4012 above the basket 4006. The customer views the products in the basket 4006 through the clear display 4012. In order to accommodate customers having different heights and postures, the body portion 4010 supports the support portion 4013 so that the support portion 4013 can move up and down, and the support portion 4013 can swing about the horizontal axis when the clear display 4012 is viewed from the front direction. To support. By raising and lowering the support portion 4013 and swinging the clear display 4012, the customer can narrow or widen the distance from the basket 4006 to the clear display 4012. When a customer moves a large product from the basket 4006 to the bag 4019, the distance from the basket 4006 to the clear display 4012 may be increased so that the product or hand does not hit the clear display 4012. The customer may reduce the distance from the basket 4006 to the clear display 4012 for easy viewing.

The image pickup unit 4014 includes a CCD (Charge Coupled Device) image pickup device which is an area image sensor and a drive circuit thereof, and an image pickup lens for forming an image of an image pickup region on the CCD image pickup device. The imaging area is set as a range over which the products in the basket 4006 can be seen from above the basket 4006 placed on the basket scale 4011. The image pickup unit 4014 outputs an image of the image pickup area formed on the CCD image pickup element through the image pickup lens.

FIG. 41 is a block diagram showing the hardware configuration of the commodity reading device 1 and the POS terminal 2. The commodity reading device 1 is equipped with a CPU (Central Processing Unit) 4101. The ROM 4103 and the RAM 4104 are connected to the CPU 4101 via a bus line 4102 such as an address bus and a data bus. A program executed by the CPU 4101 is stored in the ROM 4103. The RAM 4104 stores data of the object data file 7 transmitted from the POS terminal 2, sales registration list data, and total variables for saving the total value of the weights of the basket scale 4011 and the bag scale 4016. The image pickup unit 4014, the basket scale 4011, the clear display 4012, the bag scale 4013, and the connection interface 4105 are connected to the bus line 4102. The CPU 4101 communicates with the POS terminal 2 via the connection interface 4105. Since the POS terminal 2 has the same configuration as that of the first embodiment, detailed description will be omitted.

FIG. 42 is a flowchart showing the processing of the CPU 4101 of the commodity reading device 4001.

The CPU 4101 starts processing in response to a signal generated when the basket 4006 is placed on the basket scale 4011.

The CPU 4101 determines whether the weight indicated by the basket scale 4011 is equal to or greater than the weight of the empty basket 4006 alone (4200). If the weight indicated by the basket scale 4011 is smaller than the weight of the empty basket 4006 alone (No in 4200), the CPU 4101 ends the process. If the weight indicated by the basket scale 4011 is equal to or more than the weight of the empty basket 4006 alone (Yes in 4200), the CPU 4201 is executed. The CPU 4101 sets the processing start flag to 0 (4201).

The CPU 4101 stores the total of the weight indicated by the basket scale 4011 and the weight indicated by the bag scale 4016 in the total variable (4202). The CPU 4101 captures an image of the image capturing area with the image capturing unit 4014 (4203). The CPU 4101 saves the image of the imaging area in the RAM 4104. An image of an imaging area as a result of the previous imaging is referred to as a previous image G1, and an image of an imaging area as a result of this time imaging is referred to as a current image G2.

When the processing start flag is 0 (No in 4204), the CPU 4101 performs 4206 without performing 4205. When the processing start flag is 1 (Yes in 4204), the CPU 4101 searches the RAM 4104 for a current image G2 that is different from the previous image G1 in the RAM 4104 (4205). If there is no different part on the current image G2 from the previous image G1 (No in 4205), the CPU 4101 executes 4203 again. If the current image G2 has a portion different from the previous image G1 (Yes in 4205), 4206 is performed.

The CPU 4101 searches the RAM 4104 for a portion on the current image G2 that matches the appearance feature parameter of each product defined in the object data file 7 (4206). A portion of the current image G2 on the RAM 4104 that matches the appearance feature parameter of the product is called a matching portion.

When the matching portion is not found (No in 4206), the CPU 4101 determines whether the processing start flag is 1 (4215). If the processing start flag is 1 (Yes in 4215), 4203 is performed again. If the processing start flag is 0 (No in 4215), the processing ends.

When the matching portion is found (Yes in 4206), the CPU 4101 determines whether or not the matching portion matches the basket 4006 (4207). When the matched portion is the portion matched with the basket 4006 (Yes in 4207), if the processing start flag is 0 (No in 4208), the processing ends. If the processing start flag is 1 (Yes of 4208), the CPU 4101 causes the basket 4006 to overlap the products registered in the sales registration list with the total amount, the tax amount, and the total amount to be paid by the customer, as shown in FIG. Then, the sales registration list is output to the POS terminal 2 via the connection interface 4105 (4209), and the processing is terminated.

When the matched portion is not the portion matched with the basket 4006 (No in 4207), the CPU 4101 sets the processing start flag to 1 (4210). In the configuration of this flowchart, the processing start flag indicates whether or not the CPU 4101 has started the processing while the product is in the imaging area.

The CPU 4101 determines whether the total of the weight indicated by the basket scale 4011 and the weight indicated by the bag scale 4016 matches the value stored in the total variable (4211). When the sum of the weight indicated by the basket scale 4011 and the weight indicated by the bag scale 4016 is equal to the value stored in the total variable (Yes in 4211), the CPU 4101 causes the marking image to mark the matching portion of one product. Is stored in the RAM 4104 (4212). The CPU 4101 displays the print image on the clear display 4012 using the area other than the print as a transparent color (4213). The clear display 4012 will display the mark on the transparent basket 4006. The CPU 4101 registers the merchandise with the matched portion marked in the sales registration list (4214). The CPU 4101 again executes 4203.

When the sum of the weight indicated by the basket scale 4011 and the weight indicated by the bag scale 4016 does not match the value stored in the total variable (No in 4211), the CPU 4101 performs 4203 again without performing 4212 to 4214.

The store accounting system according to the second embodiment automatically recognizes the product when the basket containing the product is installed under the imaging area, and thus it is possible to reduce the time loss until the start of sales registration. it can. Further, when a basket heavier than an empty basket alone is placed, recognition of the product is automatically started, so that it is possible to prevent erroneous sales registration start.

Further, the store accounting system according to the second embodiment starts recognizing the next product after confirming that the sales-registered product has moved from the basket to the bag by using the basket scale and the bag scale. , It is possible to prevent forgetting to put the product taken out of the basket in the bag.

Further, in the store accounting system according to the second embodiment, since the product and the actual basket can be seen through the clear display, the image captured by the image capturing unit does not need to be used for visual recognition, and has an image quality specialized for product recognition. Therefore, it is easy to improve the recognition rate of the product. In addition, it is possible to suppress the discomfort caused by the delay of the product image when the image is picked up by the image pickup unit and the frame rate is low and the image is transferred from the basket to the bag. Further, the display device can be simplified because it is not necessary to display the image of the product or the basket.

Although the first embodiment is a merchandise reading device operated by a store clerk, it can also be applied to a merchandise reading device of a self-checkout system in which a customer himself/herself makes a transaction. The second embodiment is a product reading device operated by a customer, but it can also be applied to a product reading device operated by a store clerk. The merchandise reading device may be incorporated in the POS terminal and integrally configured.

In each of the embodiments, the program that realizes the functions of the invention is pre-recorded in the ROM that is the program storage unit inside the device. However, the present invention is not limited to this, and a similar program may be downloaded to the device from a network. Good. Alternatively, a similar program recorded on a recording medium may be installed in the device. The recording medium may be in any form as long as it can store the program and can be read by the device, such as a CD-ROM or a memory card. The function obtained by installing or downloading the program may be one that realizes the function in cooperation with an OS (operating system) inside the apparatus.

Besides, some embodiments of the present invention have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope equivalent thereto.

DESCRIPTION OF SYMBOLS 1... Product reading device, 10A... Body part, 10B... Eaves part, 11... Keyboard, 12... Touch panel, 13... Customer display, 14... Imaging part, 101 ... CPU, 102... Bus line, 103... ROM, 104... RAM, 105... Connection interface, 106... Connection interface, 121... Panel display unit, 122... Touch panel sensor, 2... POS terminal, 21... keyboard, 22... operator display, 23... customer display, 24... receipt printer, 201... CPU, 202... bus Line, 203... ROM, 204... RAM, 205... HDD device, 206... Communication interface, 207... Connection interface, 3... Accounting counter, 4... Cash register stand, 5 ...Drawer, 6... Basket, SC... Store computer, 7... Object data file, 8... Sales registration list, 4001... Merchandise reading device, 4010... Body part, 4011・・・ Basket scale, 4012 ・・・ Clear display, 4013 ・・・ Support section, 4014 ・・・ Imaging section, 4015 ・・・ Bagging stand, 4016 ・・・ Bag scale, 4017 ・・・ Bag rod, 4018 ... hook 4101... CPU, 4102... bus line, 4103... ROM, 4104... RAM, 4006... basket, 4019... bag.

Claims (6)

An image capturing unit that captures an image of the container containing the product,
This imaging unit
The first image taken,
A second image captured after the first image;
A third image captured after the second image,
A storage unit for storing
The first image and the second image stored in the storage unit have different portions,
When a plurality of products can be identified on the second image, sales of each of the identified products are registered,
If any of the products that can be identified on the second image and registered for sales cannot be identified on the third image, a product different from the products that can be identified on the second image and registered for sales is displayed. A processing unit that registers sales of newly identified products,
A device for reading merchandise. The processing unit outputs a list of sales-registered products when only the container is identified from the product and the container on an image captured after the first image. Commodity reading device. The product reading device according to claim 1, wherein the storage unit stores a position of the product registered for sale on an image captured after the first image. The product reading device according to claim 1, further comprising a display unit that marks the product registered for sale by the processing unit and displays an image captured after the first image. An image capturing unit that captures an image of the container containing the product,
A storage unit that stores the captured image;
In the processing unit of the computer equipped with
A function of storing the first image in the storage unit,
A function of storing a second image captured after the first image in the storage unit,
A function of storing a third image captured after the second image in the storage unit,
The first image and the second image stored in the storage unit have different portions,
When a plurality of products can be identified on the second image, sales of each of the identified products are registered,
If any of the commodities that can be identified on the second image and registered for sales cannot be identified on the third image, it is different from the product that can be identified on the second image and registered for sales. A function to register sales of newly identified products,
A program for realizing. An image capturing unit that captures an image of the container containing the product,
This imaging unit
The first image taken,
A second image captured after the first image;
A third image captured after the second image,
A storage unit for storing
The first image and the second image stored in the storage unit have different portions,
When a plurality of products can be identified on the second image, sales of each of the identified products are registered,
If any of the commodities that can be identified on the second image and registered for sales cannot be identified on the third image, it is different from the product that can be identified on the second image and registered for sales. A processing unit that registers sales of newly identified products,
An accounting processing unit that performs accounting processing of the price of the sales-registered product,
Store accounting system with.

Images