JP2020149450A - Commodities management system and method for managing commodities - Google Patents

Abstract

To provide a commodities management system and a method for managing commodities which can automatically detect a return by mistake of a commodity on a store shelf.SOLUTION: A commodities management system 1 is for managing commodities in store shelves, the store shelves displaying the same type of commodities in one row, and includes: a camera sensor 11 for acquiring a front image of a commodity on the front top position of each row of a store shelf; an identification controller 10 according to a commodity type for generating commodity front surface information which identifies the type and the front position of a commodity in the front of the store shelf on the basis of the front image; a panel sensor S on a store shelf for acquiring a bottom image of a commodity on the store shelf; a panel sensor controller 20 for generating store shelf information which identifies the size of commodities on each store shelf and the number of commodities of each row and the position of a commodity on the store shelf according to the bottom image; and a commodity management controller 30 having a wrong return detection unit for detecting a return of a commodity by mistake to a position which is not specified, on the basis of the commodity front surface information and the store shelf information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a product management system and a product management method capable of automatically detecting an erroneous return of a product placed on a product shelf.

In stores such as convenience stores and supermarkets, products are often displayed in showcases in which multiple product shelves are arranged in multiple stages. The store needs to manage the replenishable number of products placed on the product shelves and replenish the products in a timely and appropriate manner.

In addition, Patent Document 1 describes a system that uses a motion sensor and a weight sensor to determine a product that has been taken out or a product that has been placed based on the detection timing. Further, Patent Document 2 describes a product status analyzer that detects that an object has disappeared from the object placement area and that the object has returned from the product placement area based on the image information obtained by the camera. Has been done.

Japanese Unexamined Patent Publication No. 2018-206372 Japanese Unexamined Patent Publication No. 2014-149678

By the way, the products placed on the product shelves of the showcase are placed in the positions specified in advance, but there is an erroneous return in which the products once taken out by the customer are returned to a position other than the specified position. Since the clerk patrolled the store and visually confirmed the product that was returned by mistake, there was a problem that the work burden on the clerk was heavy.

It should be noted that the erroneous return is a phenomenon in which the removal and return of the product are combined, and it is difficult to detect the erroneous return only by simply detecting the product.

The present invention has been made in view of the above, and an object of the present invention is to provide a product management system and a product management method capable of automatically detecting an erroneous return of a product placed on a product shelf.

In order to solve the above-mentioned problems and achieve the object, the product management system according to the present invention is a product on a product shelf in which products of the same product type are placed in a row from the front leading position of the product shelf toward the depth direction. A camera sensor that acquires a front image of a product placed at the front head position of each row of the product shelf, and a product on the front of the product shelf based on the front image. A front identification unit that generates product front information that identifies the product type and front position of the product, a panel sensor that is placed on the product shelf and acquires a bottom image of the product placed on the product shelf, and the product front surface. A product shelf identification unit that generates product shelf information that identifies the size of the product, the number of products in each row, and the product shelf position of the product for each product shelf based on the information and the bottom image is specified in advance. Product management with an erroneous return detection unit that detects erroneous returns when the product is returned to a position other than the specified position based on the product arrangement information, the product front information, and the product shelf information related to the arrangement of the products to be performed. It is characterized by having a part and.

Further, in the product management system according to the present invention, in the above invention, when the erroneous return detection unit is different from the product type specified in advance of the current product at the front position based on the product front information. , When the size of the products in the same column is not the same as the size of the product specified in advance based on the product shelf information, or the products with 1 or more products in the column where the number of products is 0 based on the product shelf information. When is placed, it is characterized in that it is detected as having the erroneous return.

Further, in the product management system according to the present invention, in the above invention, the erroneous return detection unit is the product type of the product in which the product type of the current product at the front position is designated in advance based on the product front information. If they are different, the product type of the product, the product type of the product different from the product, and the front position are generated as erroneous return information.

Further, in the product management system according to the present invention, in the above invention, when the size of the product in the same row is not the same as the size of the product specified in advance in the erroneous return detection unit based on the product shelf information. It is characterized in that the product type and the front position of products whose product sizes are not the same are generated as erroneous return information.

Further, in the product management system according to the present invention, in the above invention, when the erroneous return detection unit places one or more products in a column where the number of products is 0 based on the product shelf information. It is characterized in that the product type and the front position of the products in the row in which one or more products are placed are generated as erroneous return information.

Further, in the product management system according to the present invention, in the above invention, when the erroneous return detection unit detects the erroneous return, the product management unit connects the erroneous return information to the product management system. It is characterized by having an output processing unit that outputs a notification to the device.

Further, the product management method according to the present invention is a product management method for managing products on a product shelf in which products of the same product type are placed in a row extending from the front front position of the product shelf toward the depth direction. A front image acquisition step for acquiring a front image of a product placed at the front head position of each row of a product shelf, and a product that identifies the product type and front position of the product on the front of the product shelf based on the front image. The front identification step for generating the front information, the bottom image acquisition step for acquiring the bottom image of the product placed on the product shelf and placed on the product shelf, and the product front information and the bottom image are also included. A product shelf identification step that generates product shelf information that identifies the size of the product, the number of products in each row, and the product shelf position of the product for each product shelf, and product arrangement related to the arrangement of products specified in advance. It is characterized by including an erroneous return detection step of detecting an erroneous return in which the product is returned to a position other than a designated position based on the information, the product front information, and the product shelf information.

According to the present invention, since the erroneous return of the product placed on the product shelf is automatically detected, the work load of the clerk can be reduced.

FIG. 1 is a block diagram showing a configuration of a product management system according to an embodiment of the present invention. FIG. 2 is a schematic view showing the configuration of showcases arranged in the store. FIG. 3 is a diagram showing an example of a state in which the same product type is placed in each row of the product shelf. FIG. 4 is a diagram showing an imaging region of a front image captured by the camera sensor. FIG. 5 is a perspective view showing the configuration of a product shelf in which the panel sensor is arranged on the upper surface. FIG. 6 is a diagram showing an example of a bottom image. FIG. 7 is a flowchart showing a product erroneous return management processing procedure by the product management system. FIG. 8 is a flowchart showing the erroneous return detection processing procedure in step S105. FIG. 9 is a diagram showing an example of a case where the product type of the current product at the front position is different from the product type specified in advance. FIG. 10 is a diagram showing an example in which the sizes of products in the same row are not the same as the sizes of products specified in advance. FIG. 11 is a diagram showing an example of a case where one or more products are placed in a column having 0 products. FIG. 12 is a diagram showing an example in the case where it is determined that there is no erroneous return. FIG. 13 is a diagram showing an example in which erroneous return information is notified and output to the tablet terminal.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a block diagram showing a configuration of a product management system 1 according to an embodiment of the present invention. Further, FIG. 2 is a schematic view showing the configuration of the showcase 2 arranged in the store. As shown in FIG. 1, in the product management system 1, a product type identification controller 10 as a front identification unit, a panel sensor controller 20 as a product shelf identification unit, and a product management controller 30 are connected to the network N1. The PC terminal 40 and the tablet terminal 50 are connected to the network N1 via the network N2.

A camera sensor 11 and a storage unit 12 are connected to the product type identification controller 10. The camera sensor 11 is an imaging device that acquires the front image D1 of the showcase 2. The storage unit 12 stores the front image D1, the product feature information D2, and the product front information D3.

The product type identification controller 10 generates product front information D3 that identifies the product type and the front position (XY plane) of the product in each row of the product shelf based on the front image D1. At this time, the product type identification controller 10 identifies the product type of the front image D1 with reference to the product feature information D2 which is the feature information of the front image for each product type. The product type identification controller 10 sends the generated product front information D3 to the panel sensor controller 20. The product feature information D2 is information indicating a product packaging state including, for example, a product color, pattern, design, and shape.

As shown in FIG. 2, in the showcase 2, the product P is placed on the plurality of product shelves 2a to 2f via the panel sensors S1 to S6, respectively. As shown in FIG. 3, on the product shelves 2a to 2f, products P of the same product type are placed in each row R from the front front position toward the depth direction (+ Z direction).

As shown in FIG. 2, the camera sensor 11 is attached to the ceiling 101, for example, and acquires the front image D1 of the product shelves 2a to 2f. As shown in FIG. 3, when the product P exists in each column R, the product P appears on the front surface. As shown in FIG. 4, the camera sensor 11 acquires an image for the front area E of the showcase 2 as the front image D1. The area E is an area in which the products P on all the product shelves 2a to 2f can be imaged. The front image D1 may be a set of front images divided into product shelves 2a to 2f.

In FIG. 1, a panel sensor S (S1 to S6) and a storage unit 22 arranged on a product shelf are connected to the panel sensor controller 20 via a hub 21. As shown in FIG. 2, the panel sensors S1 to S6 are arranged on the upper surfaces of the product shelves 2a to 2f. For example, as shown in FIG. 5, the panel sensor S1 is arranged on the upper surface of the product shelf 2a.

The panel sensor S is a capacitance type panel sensor, detects a change in capacitance due to a product P arranged on the upper surface of the panel sensor S in a matrix, and displays a two-dimensional image of the change in capacitance. It is acquired as the bottom image D4 of the product as shown in 6. FIG. 6 shows a bottom image D4 when the PET bottle is arranged, and a circular image corresponding to the shape of the bottom of the PET bottle is formed in the area where the PET bottle is placed.

In FIG. 1, the storage unit 22 stores the product front information D3, the bottom image D4, and the product shelf information D5.

The panel sensor controller 20 is based on the product front information D3 and the bottom image D4, the product size for each product shelf 2a to 2f, the number of products for each product type in each row R, and the product shelf position (XZ). The product shelf information D5 that identifies the plane) is generated.

As for the number of products, the template matching process of the bottom image D4 is performed using the template matching information for each product type indicated by the product front information D3, and the number of locations determined to match in the template matching process is detected as the number of products.

The product management controller 30 has an erroneous return detection unit 30a and an output processing unit 30b, and a storage unit 31 is connected to the product management controller 30. The storage unit 31 stores the product placement information D6, which is the product placement information designated in advance, and the erroneous return information D7, which is the erroneous return detection result. The product management controller 30 also manages the excess / deficient number of products based on the product front information D3 and the product shelf information D5.

The erroneous return detection unit 30a detects an erroneous return in which the product is returned to a position other than the designated position based on the product arrangement information D6, the product front information D3, and the product shelf information D5, and stores it as the erroneous return information D7. .. When the erroneous return detection unit 30a detects the erroneous return, the output processing unit 30b notifies and outputs the erroneous return information D7 to the PC terminal 40, the tablet terminal 50, and the like.

When the panel sensor controller 20 does not detect a product in the area where the first product in each column R is arranged, the product management controller 30 does not have the first product in the column R in which the product is not detected. Upon receiving this notification, the product management controller 30 notifies, for example, the tablet terminal 50 of the product advance instruction in the column R in which the product has not been detected. Further, the product management controller 30 notifies the tablet terminal 50 of the product advance instruction when one or more products are detected in the column R in which the product is not detected.

<Product misreturn management process>
Next, the product erroneous return management processing procedure by the product management system 1 will be described with reference to the flowchart shown in FIG. As shown in FIG. 7, first, the product management controller 30 acquires the product arrangement information D6 designated in advance in the storage unit 31 (step S101). After that, the product type identification controller 10 generates the product front information D3 (step S102). Further, the panel sensor controller 20 generates the product shelf information D5 (step S103).

The product management controller 30 refers to the product placement information D6, the product front information, and the product shelf information, and determines whether or not there is a product placement different from the product placement information D6 (step S104). The different product arrangement means that there is no product at the position indicated by the product arrangement information D6, there is a different product, or the product exists at a position where there is no product, and erroneous return is not detected at this point.

If there is a product arrangement different from the product arrangement information D6 (step S104, Yes), the erroneous return detection unit 30a performs the erroneous return detection process (step S105). As a result, it is determined whether or not there is an erroneous return (step S106). When an erroneous return is detected (step S106, Yes), the output processing unit 30b notifies and outputs the erroneous return information D7 to the PC terminal 40 and the tablet terminal 50 (step S107), and proceeds to step S108.

On the other hand, if there is no product arrangement different from the product arrangement information D6 (step S104, No), and if no erroneous return is detected (step S106, No), the process proceeds to step S108 as it is. In step S108, it is determined whether or not there is an end instruction (step S108), and if there is no end instruction (step S108, No), the process proceeds to step S101 and the series of processes described above is repeated, and the end instruction is issued. If there is (step S108, Yes), this process ends.

<Error return detection process>
FIG. 8 is a flowchart showing the erroneous return detection processing procedure in step S105. First, the erroneous return detection unit 30a determines whether or not the product type of the current product at the front position is different from the product type specified in advance based on the product front information D3 (step S201). If the product type of the current product at the front position is different from the product type specified in advance (steps S201, Yes), it is determined that there is an erroneous return, erroneous return information D7 is generated (step S204), and step S105. Return to.

On the other hand, when the product type of the current product in the front position is the same as the product type specified in advance (steps S201, No), the size of the products in the same row is further increased based on the product shelf information D5. It is determined whether or not the size of the product is the same as that of the product specified in advance (step S202). If the size of the products in the same column is not the same as the size of the products specified in advance (steps S202, No), it is determined that there is an erroneous return, erroneous return information D7 is generated (step S204), and step S105. Return to.

When the size of the products in the same row is the same as the size of the products specified in advance (steps S202, Yes), one or more products are added to the row with 0 products based on the product shelf information D5. It is determined whether or not it is mounted (step S203). When one or more products are placed in the column where the number of products is 0 (step S203, Yes), it is determined that there is an erroneous return, erroneous return information D7 is generated (step S204), and in step S105. Return.

On the other hand, when one or more products are not placed in the column where the number of products is 0 (step S203, No), it is determined that there is no erroneous return (step S205), and the product returns to step S105.

<Example when the product type of the current product in the front position is different from the product type specified in advance>
Here, an example in which it is determined that there is an erroneous return in step S201 will be described with reference to FIG. In this determination, the product front information D3 is referred to.

FIG. 9A is a schematic view corresponding to the product shelf information D5 on the XZ plane showing an example in which the products are arranged in three rows, and the information regarding the products in the front surface of FIG. 9A is the product front surface. Corresponds to information D3. In FIG. 9A, three products A are placed in the row R1, three products B are placed in the row R2, and no goods are placed in the row R3. The product B has a bottom image larger than that of the product A.

In FIG. 9B, in the mounting state ST10 before the false return detection, three products A1, A2, and A3 are mounted in the row R1, three products B are mounted in the row R2, and the row R3 has three products B. The product is not placed. On the other hand, in the mounting state ST11 after the erroneous return detection, the product A3 mounted on the front surface of the row R1 disappears, and the product A3'corresponding to the product A3 is mounted on the front surface of the row R3. Column R3 is a column without products, and the presence of product A3'in column R3 means that products of different product types are placed in column R3, and product A3'is an erroneously returned product. , It is judged that there is an erroneous return.

Further, in the mounting state ST12 after the erroneous return is detected, all the products A1, A2, and A3 in the row R1 disappear, and the products A1', A2', and A3' corresponding to the products A1, A2, and A3 are placed in the row R3. It is placed. Column R3 is a column without products, and the presence of product A3'in column R3 means that products of different product types are placed in column R3, and product A3'is an erroneously returned product. , It is judged that there is an erroneous return.

In FIG. 9C, in the mounting state ST20 before the false return detection, three products A1, A2, and A3 are placed in the row R1, and two products B are placed in the row R2 in addition to the front surface. ing. On the other hand, in the loading state ST21 after the erroneous return detection, the product A3'corresponding to the product A3 is mounted on the front surface of the row R2. Since the row R2 is a row in which the product B is placed and the product A3'is a different product type from the product B, the product A3'is an erroneously returned product, and it is determined that there is an erroneous return.

Further, in the loading state ST22 after the erroneous return detection, the product A1'corresponding to the product A1 other than the front surface of the row R1 is mounted on the front surface of the row R2. Since the row R2 is a row in which the product B is placed and the product A1'is a different product type from the product B, the product A1'is an erroneously returned product, and it is determined that there is an erroneous return.

In FIG. 9D, in the mounting state ST30 before the false return detection, three products A1, A2, and A3 are placed in the row R1, and one product B is placed in front of the row R2, and the row R3. The product is not placed. On the other hand, in the mounting state ST31 after the erroneous return is detected, the product A1'corresponding to the product A1 is arranged in front of the row R3. Since the product A1'is a different product type from the one without the product, the product A1'is a product of erroneous return, and it is determined that there is erroneous return.

<Example when the size of products in the same row is not the same as the size of products specified in advance>
Here, an example in which it is determined that there is an erroneous return in step S202 will be described with reference to FIG. In this determination, the product shelf information D5 is referred to.

In FIG. 10, three products A1, A2, and A3 are placed in the row R in the mounting state ST40 before the false return is detected, and the product B is placed in front of the row R2. On the other hand, in the loading state ST41 after the erroneous return detection, the products A1'and A2' corresponding to the products A1 and A2 other than the front surface of the row R1 are mounted at positions other than the front surface of the row R2. Since the products A1'and A2'have different sizes from the product B in the row R2, the products A1'and A2' are erroneously returned products, and it is determined that there is an erroneous return.

In the mounting state ST42 after the erroneous return is detected, the product A3'corresponding to the product A3 on the front surface of the row R1 is mounted at a position other than the front surface of the row R2. Since the product A3'has a different size from the product B in the row R2, the product A3'is an erroneously returned product, and it is determined that there is an erroneous return.

<Example when one or more products are placed in a column with 0 products>
Here, an example in which it is determined in step S203 that there is an erroneous return will be described with reference to FIG. In this determination, the product shelf information D5 is referred to.

In FIG. 11, in the loading state ST50 before the false return detection, three products A1, A2, and A3 are placed in the row R1, three products B are placed in the row R2, and the products are placed in the row R3. Not placed.
On the other hand, in the loading state ST51 after the erroneous return detection, the products A1'and A2' corresponding to the products A1 and A2 other than the front surface of the row R1 are mounted at positions other than the front surface of the row R3. Since the column R3 is a column in which the number of products is 0 and one or more products A1'and A2' are placed in the row R3, the products A1'and A2' are erroneously returned products, and there is an erroneous return. Is determined.

In the mounting state ST52 after the erroneous return is detected, the product A3'corresponding to the product A3 on the front surface of the row R1 is mounted at a position other than the front surface of the row R3. Since the column R3 is a column in which the number of products is 0 and one or more products A3'are placed in the row R3, the product A3'is an erroneously returned product, and it is determined that there is an erroneous return.

<Example when it is judged that there is no erroneous return>
Here, an example in the case where it is determined that there is no erroneous return will be described with reference to FIG. In FIG. 12, in the mounting state ST60 before the false return detection, three products A1, A2, and A3 are placed in the row R1, three products B1, B2, and B3 are placed in the row R2, and the row R3. , The product is not listed. On the other hand, in the mounted state ST61 after the erroneous return is detected, the product A1 on the front surface of the row R1 is taken out and disappears. In this case, since the determination conditions of steps S201 to 203 are not met, it is determined that there is no erroneous return.

In the mounted state ST62 after the erroneous return is detected, the product A1 other than the front surface of the row R1 has disappeared and is taken out. In this case, since the determination conditions of steps S201 to 203 are not met, it is determined that there is no erroneous return.

In the mounted state ST63 after the erroneous return is detected, all the products A1, A2, and A3 in the row R1 are removed. In this case, since the determination conditions of steps S201 to 203 are not met, it is determined that there is no erroneous return.

<Example of notification output of erroneous return information>
FIG. 13 is a diagram showing an example in which the erroneous return information D7 is notified and output to the tablet terminal 50. As shown in FIG. 13, in the notification output, erroneous return information including the erroneously returned product (product type) and the erroneous return product arrangement position (front position; product shelf stage and X direction position) is displayed. The clerk who carries the tablet terminal 50 can quickly and easily return the erroneously returned product to the original product position by referring to the notification output.

It should be noted that each configuration shown in the above-described embodiment and modification is a schematic function, and does not necessarily have to be physically shown. That is, the form of distribution / integration of each device and component is not limited to the one shown in the figure, and all or part of the device and components are functionally or physically distributed / integrated in arbitrary units according to various usage conditions. Can be configured.

1 Product management system 2 Showcase 2a to 2f Product shelf 10 Product type identification controller 11 Camera sensor 12, 22, 31 Storage unit 20 Panel sensor controller 21 Hub 30 Product management controller 30a False return detection unit 30b Output processing unit 40 PC terminal 50 Tablet terminal 101 Ceiling A, A1 to A3, A1'to A3', B, B1, B2, B3 Product D1 Front image D2 Product feature information D3 Product front information D4 Bottom image D5 Product shelf information D6 Product placement information D7 False return information E area N1, N2 network P product R, R1 to R3 rows S, S1 to S6 panel sensors ST10 to ST12, ST20 to ST22, ST30, ST31, ST40 to ST42, ST50 to ST52, ST60 to ST63

Claims (7)

It is a product management system that manages products on the product shelves in which products of the same product type are placed in rows from the front top position of the product shelves to the depth direction.
A camera sensor that acquires a front image of a product placed at the front head position of each row of the product shelf,
A front identification unit that generates product front information that identifies the product type and front position of the product on the front of the product shelf based on the front image.
A panel sensor that is placed on the product shelf and acquires a bottom image of the product placed on the product shelf,
A product shelf identification unit that generates product shelf information that identifies the size of the product, the number of products in each row, and the product shelf position of the product for each product shelf based on the product front information and the bottom image. ,
It has an erroneous return detection unit that detects an erroneous return in which the product is returned to a position other than the specified position based on the product arrangement information regarding the arrangement of the product specified in advance, the product front information, and the product shelf information. Product management department and
A product management system characterized by being equipped with. When the product type of the current product at the front position is different from the product type specified in advance based on the product front information, the erroneous return detection unit determines the size of the products in the same row based on the product shelf information. If the size is not the same as the size of the product specified in advance, or if one or more products are placed in the column where the number of products is 0 based on the product shelf information, it is detected as having the erroneous return. The product management system according to claim 1. If the product type of the current product at the front position is different from the product type of the product specified in advance based on the product front information, the erroneous return detection unit may use the product type and front of the product different from the product type. The product management system according to claim 2, wherein the position is generated as erroneous return information. When the size of the products in the same row is not the same as the size of the product specified in advance based on the product shelf information, the erroneous return detection unit determines the product type and front position of the products whose product sizes are not the same. The product management system according to claim 2, wherein the product is generated as erroneous return information. When one or more products are placed in a row with 0 products based on the product shelf information, the erroneous return detection unit determines the product type of the products in the row in which one or more products are placed. The product management system according to claim 2, wherein the front position is generated as erroneous return information. The product management unit is characterized by including an output processing unit that, when the erroneous return detection unit detects the erroneous return, notifies and outputs the erroneous return information to a terminal device connected to the product management system. The product management system according to any one of claims 3 to 5. It is a product management method that manages products on the product shelves in which products of the same product type are placed in rows from the front top position of the product shelves to the depth direction.
A front image acquisition step for acquiring a front image of a product placed at the front head position of each row of the product shelf, and a front image acquisition step.
A front identification step that generates product front information that identifies the product type and front position of the product on the front of the product shelf based on the front image, and
A bottom image acquisition step for acquiring a bottom image of a product placed on the product shelf and placed on the product shelf,
A product shelf identification step that generates product shelf information that identifies the size of the product, the number of products in each row, and the product shelf position of the product for each product shelf based on the product front information and the bottom image. ,
An erroneous return detection step that detects an erroneous return in which the product is returned to a position other than the specified position based on the product arrangement information regarding the arrangement of the product specified in advance, the product front information, and the product shelf information.
A product management method characterized by including.

Images