JP7040126B2 - Inventory monitoring system - Google Patents

Description

The present invention relates to an inventory monitoring system, and is suitable for being applied to an inventory monitoring system that monitors an inventory of products by analyzing image data obtained by taking a picture of a product display unit with a camera.

Conventionally, a barcode scanner in a POS terminal device operated by a store staff reads the product code of a product purchased by a customer, inputs sales data including sales quantity, and stores POS as sales data for each sales item in a database. A system has been proposed (see, for example, Patent Document 1).

In addition, by providing a system that enables inventory monitoring, ordering, and settlement of standard products in the refrigerators of each household, what kind of lifestyle households have a share of each product to be sold, etc. It has been proposed that detailed data can be obtained and the accuracy is higher than that of analysis utilizing a huge amount of POS data, and that product renewal and target strategy can be performed more effectively (see, for example, Patent Document 2). ..

Japanese Unexamined Patent Publication No. 8-22576 JP-A-2017-68812

The technique of simply monitoring inventory as described above has been conventionally known. However, in the case of oden and hot snacks (fried foods such as croquettes) at convenience stores and other stores, they are sold after preparing for provision of products such as warming in the store, so they are simply inventory monitoring that has been known in the past. However, there is a problem that inventory monitoring cannot be performed in consideration of product provision preparation and sales opportunities.

The present invention has been made in consideration of the above points, and an object of the present invention is to propose an inventory monitoring system capable of performing inventory monitoring in consideration of product provision preparation and sales opportunities.

In order to solve such a problem, in the inventory monitoring system of the present invention, an imaging means for imaging a monitoring target, an analysis means for analyzing an image captured by the imaging means to determine the loading state of the monitoring target, and a determination of the loading state. A notification means for notifying the user of the number of monitoring targets to be replenished according to the result is provided.

Thereby, the present invention automatically notifies the user that the product should be replenished based on the captured image of the product without any operation of the user, and informs the user of the product at the timing of selling the product. Can be replenished and prepared for provision.

According to the present invention, it is possible to realize an inventory monitoring system capable of performing inventory monitoring in consideration of product provision preparation and sales opportunity.

It is a figure which shows the structure of the inventory monitoring system. It is a perspective view which shows the structure of an oden sales department. It is a block diagram which shows the structure of a tablet terminal. It is a block diagram which shows the functional structure of a tablet terminal. It is a flowchart which shows the loading state analysis processing procedure. It is a flowchart which shows the replenishment notification processing procedure. It is a flowchart which shows the loading state notification processing procedure. It is a figure which shows the display product image pickup screen. It is a figure which shows the previous display product image pickup screen. It is a figure which shows the egg image. It is a figure which shows the display product image pickup screen in the state which the replenishment quantity notification part is displayed. It is a figure which shows the display product image pickup screen in the state which the replenishment notification part is displayed. It is a figure which shows the display product image pickup screen in the state which the ingredient is selected. It is a figure which shows the display product image pickup screen in the state which the loading quantity sufficiency notification part is displayed. It is a figure which shows the sales quantity table by egg day. It is a figure which shows the sales quantity table by egg time.

Hereinafter, embodiments for carrying out the invention (hereinafter referred to as embodiments) will be described with reference to the drawings.

[1. Embodiment]
[1-1. Inventory monitoring system configuration]
As shown in FIG. 1, the inventory monitoring system 1 is composed of a tablet terminal 2, a surveillance camera 4, a server 6, a wireless antenna 8, an oden display unit 10, and a lunch box display unit 12, for example, in a store ST such as a convenience store. It is installed in.

The tablet terminal 2 as an information processing terminal is arranged in the oden sales unit 10S shown in FIG. 2, is operated by a salesperson, that is, a store clerk, and the control unit 20 controls and controls the whole as shown in FIG. The control unit 20 is mainly composed of a CPU (Central Processing Unit), and reads a predetermined program from a storage unit 24 including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, a flash memory, and the like. By executing the above, each part is controlled and the inventory monitoring operation is performed.

The control unit 20 has an existing height monitoring processing unit, an existing height management processing unit, a camera switching processing unit, and an operation display processing unit (not shown). The existing height monitoring processing unit performs image analysis on the image data of the product such as oden received from the surveillance camera 4 and monitors the existing height of the product. The existing amount management processing unit manages the existing amount. The camera switching processing unit switches the subject of the surveillance camera 4. The operation display processing unit processes the screen display of the operation display unit 22. The operation display unit 22 integrates an LCD (Liquid Crystal Display) that displays a screen for giving various instructions and operation guidance, a display product imaging screen, etc., which will be described later, and a touch panel for selecting various instructions and inputting various information. It has been transformed.

The storage unit 24 has an image of each ingredient IG (FIG. 2) of oden, an image of the oden display / sale case 10A, and a loading area 10B (loading area 10B) in which each ingredient IG in the oden display / sale case 10A is loaded. The position of FIG. 2) is learned and stored in advance. Further, the storage unit 24 stores the current loading quantity for each ingredient IG of oden as information on the loading quantity for each ingredient, and aggregates the sales quantity for each ingredient IG of oden in chronological order to sell the quantity for each ingredient. Store as information. The camera communication unit 26 is, for example, an interface for a LAN (Local Area Network) cable, and performs data communication with the surveillance camera 4. The server communication unit 28 is, for example, an interface for a wireless LAN, and performs data communication with the server 6.

The surveillance camera 4 (FIG. 1) is connected to the tablet terminal 2 by, for example, a LAN cable. For example, the oden display unit 10 is imaged and the image data is transmitted to the tablet terminal 2. The surveillance camera 4 is arranged at a position where the height of ingredients such as oden can be discriminated, and has a resolution at which the height of ingredients can be discriminated. Further, the surveillance camera 4 has a resolution that can be discriminated by enlarging the lunch box or the like to which the bar code is attached. Further, when there are a plurality of subjects, the surveillance camera 4 automatically switches the subjects or based on the instruction of the control unit 20.

The server 6 is connected to the tablet terminal 2 by a wireless LAN such as Wi-Fi (registered trademark), wirelessly communicates with the tablet terminal 2, and is connected to a predetermined base outside the store ST by a network. Various information received from the tablet terminal 2 is accumulated and the information is analyzed. The server 6 holds the egg daily sales quantity table TB1 shown in FIG. 15 and the egg hour sales quantity table TB2 shown in FIG. 16 in the storage unit, and forecasts sales data in response to a request from the tablet terminal 2. The sales forecast information is transmitted to the tablet terminal 2.

As shown in FIG. 2, the oden display unit 10 has an oden ingredient IG such as an egg or a chikuwa for each loading area 10B in the oden display / sales case 10A in which the loading area 10B is divided for each type of ingredient IG. Is stored. The bento display unit 12 is arranged so that a plurality of bento LBs are overlapped for each type, and the container CL of the bento LB has a two-dimensional code TDC, a bar code BC, and an RFID, which are unique to each type of bento LB. Radio Frequency Identifier) tag TG is attached. In this way, in the lunch box display section 12, a plurality of tagged products with the RFID tag TG are arranged for each type of lunch box LB.

The wireless antenna 8 is a highly directional RFID reader, and can read the RFID tag TG of the bento LB, which is a tagged product in the bento display unit 12, for example. The control unit 20 of the tablet terminal 2 analyzes the image data of the lunch box display unit 12 received from the surveillance camera 4 to detect the arrangement relationship of the lunch box LB, and the RFID tag TG of the lunch box LB by the beamforming function of the wireless antenna 8 is used. By reading the above, it is possible to specify the display state indicating how many lunch boxes LB of which type are displayed.

[1-2. Composition of Oden Sales Department]
An oden sales department 10S where oden is sold is provided behind the convenience store, which is on the side away from the clerk from the cashier counter. In the following, the side of the Oden Sales Department 10S that the clerk faces is the front side, and the opposite, that is, the side that the customer faces is the rear side, and the left, right, top, and bottom when viewed from the clerk facing the front side, respectively. It will be described as left side, right side, upper side and lower side.

The oden sales unit 10S has a table 10T, a heat insulating device 10H, an oden display / sales case 10A, a lid 10C, a tablet terminal 2, and a heat blocking panel 10HS. The heat insulating device 10H has a box shape having a predetermined thickness, and two units are placed side by side on the table 10T in the left-right direction and connected to a commercial power source, respectively, and the oden display / sales case 10A is heated by a heater.

Each of the oden display and sales cases 10A is placed on the upper surface of the heat insulating device 10H on the rear side away from the clerk standing at the cash register counter, and stores various oden ingredients IG. The oden display / sales case 10A is formed to have a shorter length in the front-rear direction than the heat insulating device 10H. Therefore, there is an empty space in front of the oden display / sales case 10A on the upper surface of the heat insulating device 10H.

The lid portion 10C is a lid configured to cover the oden display / sales case 10A, and by being placed on the upper part of the oden display / sales case 10A, the oden ingredients stored in the oden display / sales case 10A can be removed from the outside. On the other hand, when the oden ingredients are taken by the customer or replenished by the clerk, they are removed from the oden display and sales case 10A. The surveillance camera 4 captures an image of oden when the lid portion 10C is removed from the oden display / sales case 10A.

The tablet terminal 2 is placed on the upper surface of the heat insulating device 10H on the front side close to the clerk at the cash register counter so that the operation display unit 22 faces the clerk and the operation display unit 22 faces slightly upward from the front. For example, it is connected to the heat insulating device 10H by a USB (Universal Serial Bus) cable 10U and supplied with power.

The heat blocking panel 10HS is a plate-shaped member made of, for example, aluminum or the like, and has a length in the left-right direction longer than the width in the left-right direction of the tablet terminal 2. It is arranged so as to extend vertically and horizontally between them. As a result, the heat blocking panel 10HS suppresses the heat generated by the heat insulating device 10H and the oden display / sales case 10A from being transmitted to the tablet terminal 2, and prevents the tablet terminal 2 from malfunctioning.

[1-3. Function configuration of tablet terminal]
Here, the basic functions related to the loading state analysis processing, the replenishment notification processing, and the loading state notification processing, which will be described later, in the tablet terminal 2 are represented by a functional block diagram as shown in FIG. In FIG. 4, the control unit 20 realizes each functional block of the analysis means 40 and the notification means 42 by executing a predetermined loading state analysis program, replenishment notification program, and loading state notification program. The analysis means 40 analyzes the captured image captured by the surveillance camera 4 and determines the loading state of the ingredients. The notification means 42 notifies the clerk that the ingredients should be replenished according to the determination result of the loading state.

[1-4. Loading state analysis processing procedure]
Next, a specific processing procedure of the loading state analysis processing by the tablet terminal 2 will be described with reference to the flowchart of FIG. The control unit 20 starts the loading state analysis processing procedure RT1 by reading the loading state analysis processing program from the ROM and executing it, and moves to step SP1. In step SP1, the control unit 20 starts tracking the change in the captured image, thereby tracking the change in the captured image at regular intervals or constantly, and moves to step SP2.

In step SP2, the control unit 20 receives image data indicating the image captured by the oden display unit 10 from the surveillance camera 4, and proceeds to step SP3. In step SP3, the control unit 20 displays the display product imaging screen DIP shown in FIG. 8, which is an image captured by the oden display unit 10, on the operation display unit 22 based on the image data received in step SP2, and proceeds to step SP4. Move. In the display product imaging screen DIP, although the oden display and sale case 10A is shown, the table 10T, the heat insulating device 10H, the heat blocking panel 10HS, and the tablet terminal 2 are not shown and omitted.

In step SP4, the control unit 20 compares and analyzes the current image data with the previous image data to determine which oden ingredient has been sold to the customer and the number of loaded pieces has decreased, or the store clerk has replenished and loaded the oden. It is detected whether the number has increased, and the process proceeds to step SP5. Here, it is assumed that the image data this time is the display product imaging screen DIP shown in FIG. 8, and the previous image data is the display product imaging screen DIP shown in FIG. At this time, the control unit 20 is based on, for example, the egg shape image IGE shown in FIG. 10 previously learned in the storage unit 24 and the position of the loading area 10B in which the egg is loaded in the oden display / sale case 10A. It is automatically determined by image analysis that "4 pieces have been sold." Incidentally, when the step SP4 is processed for the first time, the control unit 20 skips the step SP4 because the previous image data does not exist.

Further, when the control unit 20 cannot determine the current number of the ingredient IGs loaded in detail by image analysis due to factors such as the resolution of the surveillance camera 4 being low or the shooting direction being inappropriate, the oden display is performed. By determining to some extent how much the egg is loaded in, for example, the egg loading area 10B of the sales case 10A, it is detected in two stages of whether replenishment is necessary or not. Similarly, when the store clerk replenishes the ingredient IG, the control unit 20 can accurately determine the current number of loaded units by image analysis when the resolution of the surveillance camera 4 is high, while the control unit 20 can accurately determine the current number of loads by image analysis, while the control unit 20 has a low resolution of the surveillance camera 4. Although the current number of loaded items cannot be accurately determined by image analysis, it can be determined as long as it is replenished to the extent that no further replenishment is required.

In step SP5, the control unit 20 reflects the change in the number of loaded items for each ingredient detected in step SP4 in the information on the number of loaded items for each ingredient stored in the storage unit 24, so that the information on the number of loaded items for each ingredient can be obtained. Update and move to step SP6. In step SP6, the control unit 20 totals the number of units sold for each ingredient, such as how many ingredients (products) called eggs are sold, in chronological order, and sells each ingredient stored in the storage unit 24. By reflecting it in the quantity information, the sales quantity information for each ingredient is updated, and the process returns to step SP2.

In this way, the tablet terminal 2 periodically determines that the inventory of the oden ingredients has decreased due to the sale of the oden ingredients to the customer, based on the change in the image of the display product imaging screen DIP, and for each ingredient. The number of units sold is now aggregated in chronological order. As a result, the tablet terminal 2 is easy to use by preventing the clerk from manually recording that the inventory of the oden ingredients has decreased from the oden display / sales case 10A every time the oden ingredients are sold. Can be improved.

Further, the tablet terminal 2 periodically determines that the inventory of the oden ingredients has increased due to the replenishment of the oden ingredients by the store clerk, based on the change in the image of the display product imaging screen DIP, and for each ingredient. The number of units sold is now aggregated in chronological order. As a result, the tablet terminal 2 prevents the clerk from manually recording that the inventory of the oden has increased in the oden display / sales case 10A each time the oden ingredients are replenished. , Usability can be improved.

[1-5. Replenishment notification processing procedure]
Next, a specific processing procedure for the replenishment notification process by the tablet terminal 2 will be described with reference to the flowchart of FIG. The control unit 20 starts the replenishment notification processing procedure RT2 by reading and executing the replenishment notification processing program from the ROM, and proceeds to step SP11.

In step SP11, the control unit 20 determines whether or not there is an ingredient that needs to be replenished based on the detection result in the loading state analysis processing procedure RT1 (FIG. 5) described above. If an affirmative result is obtained here, this means that there is an ingredient that needs to be replenished (hereinafter, this is also referred to as an ingredient that needs to be replenished), and at this time, the control unit 20 moves to step SP12. In the following, a case where the replenishment necessary ingredient is an egg will be described as an example. On the other hand, if a negative result is obtained in step SP11, this means that it is not necessary to replenish because the number of loaded all the ingredients is currently sufficient. At this time, the control unit 20 returns to step SP11 and replenishes. Wait until the necessary ingredients appear.

In step SP12, the control unit 20 determines whether or not the required number of replenishment required components can be recognized. If an affirmative result is obtained here, the control unit 20 moves to step SP13 at this time. In step SP13, as shown in FIG. 11, the control unit 20 replenishes four eggs by displaying the replenishment quantity notification unit 50 on the image of the egg loading area 10B which is a replenishment necessary ingredient. Prompt the clerk to do what it should, and move on to step SP14. The replenishment quantity notification unit 50 displays the type of the ingredient to be replenished and the number of ingredients to be replenished in a wording, and is displayed semi-transparently, and the image of the loading area 10B of the replenishment necessary ingredient is transmitted to the clerk. Is made visible to.

In step SP14, the control unit 20 determines that the material to be replenished has been replenished based on the detection result in the loading state analysis processing procedure RT1 (FIG. 5). If a negative result is obtained here, this means that the replenishment necessary ingredients have not been replenished by the clerk yet, and at this time, the control unit 20 displays the replenishment quantity notification unit 50 on the display product imaging screen DIP. The process returns to step SP11. On the other hand, if an affirmative result is obtained in step SP14, this means that the replenishment necessary ingredients have been replenished by the clerk, and at this time, the control unit 20 moves to step SP15. In step SP15, the control unit 20 erases the display of the replenishment quantity notification unit 50 and returns to step SP11.

On the other hand, if a negative result is obtained in step SP12, it is possible to recognize that the necessary replenishment equipment needs to be replenished due to factors such as the low resolution of the surveillance camera 4, but the number to be replenished is recognized. At this time, the control unit 20 moves to step SP16. In step SP16, as shown in FIG. 12, the control unit 20 superimposes the replenishment notification unit 52 on the image of the egg loading area 10B, which is a replenishment necessary ingredient, to indicate that replenishment is necessary. Prompt the clerk to move to step SP17. Although the replenishment notification unit 52 does not display the number to be replenished, it clearly indicates the type of the ingredient to be replenished and the fact that it should be replenished, and is displayed in a semi-transparent manner. The image of the loading area 10B of the above is transparent and visually recognized by the clerk.

In step SP17, the control unit 20 determines that the material to be replenished has been replenished based on the detection result in the loading state analysis processing procedure RT1 (FIG. 5). If a negative result is obtained here, this means that the replenishment necessary ingredients have not been replenished by the clerk yet, and at this time, the control unit 20 keeps the replenishment notification unit 52 displayed on the display product imaging screen DIP. , Return to step SP11. On the other hand, if an affirmative result is obtained in step SP17, this means that the replenishment necessary ingredients have been replenished by the clerk, and at this time, the control unit 20 moves to step SP18. In step SP18, the control unit 20 erases the display of the replenishment notification unit 52 and returns to step SP11.

As described above, when there is an ingredient whose loading quantity is not sufficient, the tablet terminal 2 superimposes the image of the loading area 10B of the ingredient on the image, and displays the quantity requiring replenishment as the replenishment quantity notification unit 50. .. As a result, the tablet terminal 2 can display the currently recognized loading state and have the store clerk confirm it, and can point out the products that need to be replenished and encourage them to replenish them.

Further, in the tablet terminal 2, if there are ingredients that are not sufficiently loaded, but the specific number to be loaded cannot be determined, the tablet terminal 2 needs to be replenished by superimposing it on the image of the loading area 10B of the ingredients. The existence is displayed as the replenishment notification unit 52. As a result, the tablet terminal 2 can point out the products that need to be replenished, and at least make the clerk recognize that the replenishment itself is necessary.

Further, since the tablet terminal 2 displays the replenishment quantity notification unit 50 or the replenishment notification unit 52 on the image of the loading area 10B of the material requiring replenishment, for example, a word such as "replenishment of 4 eggs" is added. Compared with the case of displaying in the corner inside the display product image pickup screen DIP, it is possible to make the clerk recognize at a glance which ingredient should be replenished.

[1-6. Loading status notification processing procedure]
Next, a specific processing procedure for the loading status notification process by the tablet terminal 2 will be described with reference to the flowchart of FIG. 7. The control unit 20 starts the loading status notification processing procedure RT3 by reading and executing the loading status notification processing program from the ROM, and proceeds to step SP21.

In step SP21, the control unit 20 waits until the image of the loading area 10B of the predetermined ingredient IG of oden on the display product imaging screen DIP is pressed by the clerk as shown in FIG. 13, and when pressed, the process proceeds to step SP22. Move.

In step SP22, the control unit 20 is in a state where the material selected by the clerk in step SP21 (hereinafter, this is also referred to as a high-level confirmation selection material) needs to be replenished, as in step SP11 (FIG. 6). Whether or not it is present is determined based on the detection result in the above-mentioned loading state analysis processing procedure RT1 (FIG. 5). In the following, an egg will be described as an example as a selection tool for confirming the height. If an affirmative result is obtained here, this means that the number of the height confirmation selection material is currently insufficient and needs to be replenished. At this time, the control unit 20 moves to step SP23. Incidentally, since four eggs are actually loaded as shown in FIG. 13, the control unit 20 obtains a negative result in step SP22.

In step SP23, as in step SP12 (FIG. 6), the control unit 20 determines whether or not the required number of replenishment of the height confirmation selection material can be recognized. If an affirmative result is obtained here, the control unit 20 moves to step SP24 at this time. In step SP24, as in step SP13 (FIG. 6), as shown in FIG. 11, the control unit 20 places the replenishment quantity notification unit 50 on the image of the egg loading area 10B, which is the height confirmation selection material. By displaying them in layers, the clerk is urged that four eggs should be replenished, and the process proceeds to step SP25.

In step SP25, the control unit 20 determines that the material to be replenished has been replenished based on the detection result in the loading state analysis processing procedure RT1 (FIG. 5), as in step SP14 (FIG. 6). If a negative result is obtained here, this means that the Aritaka confirmation selection material has not yet been replenished by the clerk, and at this time, the control unit 20 displays the replenishment quantity notification unit 50 on the display product imaging screen DIP. The process returns to step SP21. On the other hand, if an affirmative result is obtained in step SP25, this indicates that the high-value confirmation selection material has been replenished by the store clerk, and at this time, the control unit 20 moves to step SP26. In step SP26, the control unit 20 erases the display of the replenishment quantity notification unit 50 and returns to step SP21, as in step SP15 (FIG. 6).

On the other hand, if a negative result is obtained in step SP23, this means that it is possible to recognize that it is necessary to replenish the high-profile confirmation selection material due to factors such as the low resolution of the surveillance camera 4, but the number to be replenished. Indicates that it cannot be recognized, and at this time, the control unit 20 moves to step SP27. In step SP27, as in step SP16 (FIG. 6), as shown in FIG. 12, the control unit 20 superimposes the replenishment notification unit 52 on the image of the egg loading area 10B which is the height confirmation selection material. By displaying the image, the store clerk is urged that replenishment is necessary, and the process proceeds to step SP28.

In step SP28, the control unit 20 determines that the material to be replenished has been replenished based on the detection result in the loading state analysis processing procedure RT1 (FIG. 5), as in step SP17 (FIG. 6). If a negative result is obtained here, this means that the Aritaka confirmation selection material has not yet been replenished by the clerk, and at this time, the control unit 20 causes the replenishment notification unit 52 to be displayed on the display product imaging screen DIP. The process returns to step SP21. On the other hand, if an affirmative result is obtained in step SP28, this indicates that the high-value confirmation selection material has been replenished by the store clerk, and at this time, the control unit 20 moves to step SP29. In step SP29, the control unit 20 erases the display of the replenishment notification unit 52 and returns to step SP21, as in step SP18 (FIG. 6).

On the other hand, if a negative result is obtained in step SP22, this means that it is not necessary to replenish the loaded number of the height confirmation selection material because it is currently sufficient, and at this time, the control unit 20 goes to step SP30. Move. In step SP30, the control unit 20 displays the load quantity satisfaction notification unit 54 as shown in FIG. 14 on the display product imaging screen DIP, and proceeds to step SP31. The load quantity satisfaction notification unit 54 clearly indicates the name of the height confirmation selection material, the current load quantity, and the load quantity is sufficient. In step SP31, the control unit 20 erases the display of the loading quantity satisfaction notification unit 54 after a certain period of time has elapsed after displaying the loading quantity satisfaction notification unit 54, returns to step SP21, and the image of the predetermined ingredients of the oden is displayed by the clerk. Wait until pressed by.

In this way, when the image is pressed by the clerk and the number of selected ingredients is not sufficient, the tablet terminal 2 superimposes the image of the loading area 10B of the ingredients on the image, and the number of the replenished quantity notification unit 50 needs to be replenished. It was made to be displayed as. As a result, the tablet terminal 2 can display the currently recognized loading state and have the store clerk confirm it, and can point out the products that need to be replenished and encourage them to replenish them.

Further, since the tablet terminal 2 is configured to display the replenishment quantity notification unit 50 or the replenishment notification unit 52 on the image of the loading area 10B of the material requiring replenishment, for example, a word such as "replenishment of 4 eggs". Is displayed in the corner inside the display product imaging screen DIP, so that the store clerk can recognize at a glance which ingredient should be replenished.

Further, the tablet terminal 2 automatically erases the display of the loading quantity satisfaction notification unit 54 when a predetermined time elapses after displaying the loading quantity satisfaction notification unit 54. As a result, the tablet terminal 2 can improve usability by preventing the store clerk from manually erasing the load quantity satisfaction notification unit 54.

[1-7. Replenishment prediction method]
Here, there are two types of sales forecasts performed by the server 6, the total sales on the day, which is the total sales on the day, and the total sales on the hour, which is the total sales on the hour of the day. If the store is open 24 hours a day, the total sales by hour will be the total sales for one day. The server 6 predicts the total sales on the day and the total sales by hour by using a statistical method with reference to the number of units sold before the previous year. Further, the server 6 transmits the sales forecast information, which is the forecasted sales information, to the tablet terminal 2.

[1-7-1. How to forecast total sales on the day]
The daily sales quantity table TB1 of the egg in FIG. 15 shows the daily sales quantity of the egg. In this egg daily sales quantity table TB1, for example, 70 eggs sold on January 1, 2015, and 70 eggs sold in total from 0:00 to 23:59 on January 1, 2015. It represents that. The server 6 is based on the past data of 70, 60 and 70 units sold on January 1, 2015, 2016 and 2017, and is a general statistical method such as the Holt-Winters method. Is used to predict that the number of eggs sold on January 1, 2018 will be 62. Similarly, the server 6 predicts other dates such as January 2nd and January 3rd, 2018 based on the data of the same date in the past year.

[1-7-2. How to forecast total sales by hour]
The egg hourly sales quantity table TB2 in FIG. 16 shows the hourly sales quantity in the case of eggs. In this egg hour sales quantity table TB2, for example, the sales quantity of 5 pieces from 0:00 to 0:59 in 2015 is from 0:00 to 0: on January 1st to December 31st, 2015. It shows the average value of the number of eggs sold in 59 per day. For example, when predicting the sales of eggs from 0:00 to 0:59 in 2018, the server 6 is the number of sales from 0:00 to 0:59 in 2015, 2016 and 2017, which is the past data. Based on 5, 1, and 4, it is predicted to be 5 in the prediction column PD by using a general statistical method such as the Holt-Winters method. Similarly, the server 6 predicts other time zones such as 1:00 to 1:59 and 2:00 to 2:59 in 2018 based on the data of the same time zone in the past year.

[1-8. Judgment and prediction of whether or not replenishment is necessary]
For example, suppose that it takes two hours for the egg to permeate the taste of the soup and keep it warm before it is taken out of the refrigerator and can be provided to the customer. Then, if the current time is 22:00 on December 31, 2017, it is known that five units will be sold at 0:00 on January 1, 2018, two hours later, based on the prediction column PD. Therefore, it is necessary to secure 5 or more loading units in stock as of 22:00 on December 31, 2017. If the number of loaded eggs is less than 5 at 22:00, or if the surveillance camera 4 has a low resolution and only a rough amount of eggs can be determined, the tablet terminal 2 should have 7 eggs that are full. Encourage the clerk to replenish. As a result, the tablet terminal 2 can be completed so that the necessary and sufficient number of eggs can be provided to the customer at 0:00 on January 1, 2018.

[1-9. Effect, etc.]
In the above configuration, the inventory monitoring system 1 installs the tablet terminal 2 in a place where the store clerk can see, displays the image of the displayed product on the tablet terminal 2, and the image of the oden display unit 10 captured by the monitoring camera 4. When it is determined that the number of ingredients in stock is small by analyzing the data one by one and calculating the timing when replenishment is required by adding the preparation time that requires heat retention etc. to the ingredients in advance. The clerk is automatically notified even if it is not operated by the clerk. As a result, the inventory monitoring system 1 can appropriately prepare for the provision of the product in advance and can not miss the sales opportunity of the product.

In addition, the inventory monitoring system 1 aggregates the number of units sold for each ingredient in chronological order based on the image analysis results. Therefore, the inventory monitoring system 1 can manage the number of units sold for each ingredient with high accuracy while preventing the clerk from inputting the number of units sold.

Further, the inventory monitoring system 1 displays the replenishment quantity notification unit 50 and the replenishment notification unit 52 so as to align and superimpose the image of the loading area 10B of the material requiring replenishment on the display product imaging screen DIP. did. As a result, the inventory monitoring system 1 can make the store clerk recognize the type of ingredients to be replenished at a glance.

Further, the inventory monitoring system 1 detects an increase or decrease in the number of loaded products by analyzing the image data of the oden display unit 10 captured by the surveillance camera 4. As a result, the inventory monitoring system 1 eliminates the need for the store clerk to manually input whether the product has been replenished or sold, and can eliminate complicated work.

Here, for example, when selling oden at a convenience store, it takes a certain amount of time (2 hours, etc.) to soak the ingredients with the soup. When the sales of oden reach its peak at a certain time, if the inventory of ingredients is not sufficiently replenished before the peak, shortages will occur and opportunity loss will occur.

On the other hand, the inventory monitoring system 1 automatically manages the sales of ingredients by image processing, warns of ingredients that are likely to be out of stock during peak hours, and prompts the clerk to replenish them. Therefore, the inventory monitoring system 1 can be in a state where the ingredients are completed by the peak time and can prevent the opportunity loss from occurring. In addition, the inventory monitoring system 1 does not prompt replenishment for products that do not require replenishment. Therefore, the inventory monitoring system 1 can prevent the juice from being excessively permeated into the ingredients and the ingredients from being kept warm for a long time to be deteriorated.

In addition, sales management of ingredients can still be performed at the POS cash register, but since oden ingredients etc. are ordered in bulk for frozen foods on a weekly basis, how much sales will be in that week in advance? Needs a prediction. For this reason, if an appropriate forecast without grounds is made, the balance between purchasing and sales will be lost, and shortages and inventories will occur, resulting in loss.

On the other hand, the inventory monitoring system 1 automatically calculates the balance of ingredients. Therefore, the inventory monitoring system 1 can accumulate the number of sales on the day. In addition, the inventory monitoring system 1 can predict the number of orders per week based on the Holt Winters method or the like by using the data before the previous year. In this way, the inventory monitoring system 1 enables accurate prediction by performing automatic prediction from past results, and can prevent shortages and inventory.

Further, the inventory monitoring system 1 arranges the tablet terminal 2 in the empty space in front of the oden display / sales case 10A on the upper surface of the heat insulating device 10H. For this reason, the inventory monitoring system 1 is located in front of the oden display / sales case 10A on the upper surface of the heat insulating device 10H, which is a place that is easy for the store clerk to see and is inconspicuous to the customer in the store ST, which is a limited space. You can make effective use of empty space.

In addition, especially during lunch time, which is the so-called earning time, for example, even though a large amount of bento LB is stored in the backyard of a convenience store, a clerk enters the bento display section 12 from the backyard. Since the lunch box LB was not replenished, the lunch box LB may not be displayed in the lunch box display unit 12, or only a small number of the lunch boxes LB may be displayed. In such a case, the customer cannot purchase the undisplayed type of bento LB, and the customer tends to psychologically forgo the purchase of the bento LB in which only a small number is displayed. Therefore, not properly replenishing the lunch box LB to the lunch box display unit 12 in real time during a short lunch time will result in an extremely large opportunity loss.

On the other hand, the inventory monitoring system 1 monitors the lunch box LB of the lunch box display unit 12, and should be replenished in the image of the lunch box display unit 12 when there is a type of lunch box LB whose number of stocks on display is low. Opportunity loss can be prevented by immediately displaying on the operation display unit 22 that it should be replenished by superimposing it on the image of the type of lunch box LB.

According to the above configuration, the inventory monitoring system 1 analyzes the surveillance camera 4 that captures the ingredient IG of the oden display unit 10 as the monitoring target and the captured image captured by the surveillance camera 4, and determines the loading state of the ingredient IG. An analysis means 40 for discrimination and a notification means 42 for notifying a clerk as a user that the ingredient IG should be replenished according to the determination result of the loading state are provided.

As a result, the inventory monitoring system 1 automatically notifies the clerk that the ingredients should be replenished based on the captured image of the oden display unit 10 without any operation of the clerk, and matches the timing of selling the ingredients. It is possible to have the clerk replenish the ingredients and prepare for provision.

[2. Other embodiments]
In the above-described embodiment, when the number of loaded ingredients is currently insufficient and the required number of replenished ingredients can be recognized, the replenished quantity notification unit 50 displays the number of replenished ingredients. Said. The present invention is not limited to this, and may simply indicate that replenishment is necessary without displaying the number of replenishment. Further, the loading area 10B of the material requiring replenishment may be displayed by flicking (blinking).

Further, in the above-described embodiment, the case where the replenishment quantity notification unit 50 and the replenishment notification unit 52 are displayed so as to be aligned and overlapped with the image of the loading area 10B of the material requiring replenishment has been described. The present invention is not limited to this, and for example, a balloon or the like may be displayed so as to indicate an image of the loading area 10B of the ingredient requiring replenishment, and the wording may be displayed inside the balloon. In short, the display form should be such that the store clerk can recognize the type of ingredients to be replenished at a glance. Further, the replenishment quantity notification unit 50 and the replenishment notification unit 52 are displayed at various positions such as a corner inside the display product imaging screen DIP without aligning with the image of the loading area 10B of the material requiring replenishment. May be.

Further, in the above-described embodiment, the case where the replenishment quantity notification unit 50 is displayed on the operation display unit 22 has been described. The present invention is not limited to this, and the type of the ingredient to be replenished and the number to be replenished may be output by voice. Similarly, the replenishment notification unit 52 may output the type of the ingredient to be replenished and the fact that it should be replenished by voice. Further, either one of the type of the ingredient to be replenished and the number to be replenished may be displayed on the operation display unit 22, and the other may be output by voice. The same applies to the replenishment notification unit 52. Further, when a predetermined time has elapsed after displaying the replenishment quantity notification unit 50 or the replenishment notification unit 52 on the operation display unit 22, a voice may be output to notify the clerk.

Further, in the above-described embodiment, the case where the type of the ingredient to be replenished and the number to be replenished are displayed on the replenishment quantity notification unit 50 has been described. The present invention is not limited to this, and after aligning with the loading area 10B of the material to be replenished, only the number of the material to be replenished is sent to the replenishment quantity notification unit 50 without displaying the type of the material to be replenished. It may be displayed. Similarly, the replenishment notification unit 52 may display only that it should be replenished.

Further, in the above-described embodiment, the case where the type of the ingredient to be replenished and the number to be replenished are displayed on the replenishment quantity notification unit 50 in words has been described. The present invention is not limited to this, and for example, four eggs may be displayed side by side, and both or only the type and number of ingredients to be replenished may be displayed as an image such as a photograph instead of the wording. The same applies to the supplementary notification unit 52.

Further, in the above-described embodiment, the tablet terminal 2 performs image analysis, receives sales forecast information from the server 6, and replenishes the number notification unit 50 or the replenishment notification unit 52 based on the sales forecast information and the image analysis result. Was displayed, and the case where the loading state was determined was described. The present invention is not limited to this, and the tablet terminal 2 transmits the captured image to the server 6, the server 6 analyzes the image and determines the loading state, and transmits the identification result to the tablet terminal 2, or the control unit of the tablet terminal 2. The processing may be arbitrarily shared between the tablet terminal 2 and the server 6, such as the server 6 assisting a complicated image analysis that cannot be performed by the 20 alone. That is, as described above, the analysis means 40 may be possessed only by the tablet terminal 2, or may be possessed only by the server 6 or by both the tablet terminal 2 and the server 6.

Further, in the above-described embodiment, when it is determined by image analysis whether the number of eggs has increased or decreased based on the egg shape image IGE and the position of the loading area 10B in which the eggs are loaded in the oden display / sales case 10A. Said. The present invention is not limited to this, and for example, the images of the entire oden display / sales case 10A may be compared one by one to determine the presence or absence of each ingredient.

Further, in the above-described embodiment, the case where the analysis is performed based on the image data obtained by capturing the image of the oden display unit 10 has been described. The present invention is not limited to this, and images of various products such as meat bun display shelves, side dish display shelves, fried display shelves, vegetable display shelves, fruit display shelves, etc. that can be determined by image analysis of the actual product itself are captured. Analysis may be performed based on the data.

In addition, when the resolution of the surveillance camera 4 is high, the product is identified and the inventory is managed by image analysis of the two-dimensional code TDC and barcode BC attached to the lunch box instead of performing the image analysis of the entire lunch box. May be. In this case, the product can be specified without having to learn the image of the lunch box itself in advance. In that case, in the case of products wrapped in a predetermined wrapping paper such as confectionery and bread, and products stored in a predetermined container such as a lunch box, a two-dimensional code or barcode is attached to the wrapping paper or container. You should leave it.

Further, in the above-described embodiment, the case of monitoring two oden display / sales cases 10A has been described. The present invention is not limited to this, and one or three or more oden display / sales cases 10A, for example, one oden display / sales case 10A and one fried food display shelf may be combined at a plurality of locations at the same time. You may monitor it.

Further, in the above-described embodiment, the case where the image data captured by the surveillance camera 4 is analyzed and the replenishment number notification unit 50 and the replenishment notification unit 52 are displayed on the display product imaging screen DIP based on the image data will be described. rice field. The present invention is not limited to this, and the tablet terminal 2 itself analyzes the image data captured by the camera unit and displays the replenishment quantity notification unit 50 and the replenishment notification unit 52 on the display product imaging screen DIP based on the image data. May be.

Further, the heat insulating device 10H may measure the weight of the object placed on the upper surface. In that case, since the weight of the oden display / sales case 10A and the ingredients can be measured, the accuracy of determining the number of loaded items can be improved by comparing the increase / decrease in the number of loaded items determined by image analysis with the increase / decrease in the measured weight. At the same time, it is possible to improve the calculation accuracy of the number of units sold for each ingredient.

Further, in the above-described embodiment, the case where the heat blocking panel 10HS is made of aluminum has been described. The present invention is not limited to this, and the heat blocking panel 10HS may be made of various other materials.

Further, in the above-described embodiment, a case where the heat blocking panel 10HS is arranged between the tablet terminal 2 and the oden display / sales case 10A has been described. The present invention is not limited to this, and if the heat generated by the heat insulating device 10H and the oden display / sales case 10A does not affect the operation of the tablet terminal 2, the heat blocking panel 10HS may be omitted.

Further, in the above-described embodiment, the case where the tablet terminal 2 in which the control unit, the operation unit, and the display unit are integrated is provided has been described. The present invention is not limited to this, and the control unit, the operation unit, and the display unit are separately provided, for example, a personal computer, a mouse, and a display, and the display is arranged on the front side on the upper surface of the heat insulating device 10H to form a personal computer. A mouse may be placed around the cash register counter.

Further, in the above-described embodiment, the case where the tablet terminal 2 is arranged on the front side of the upper surface of the heat insulating device 10H has been described. The present invention is not limited to this, and the tablet terminal 2 may be arranged in various other places such as above the cash register counter. In that case, it is preferable that the tablet terminal 2 is arranged at a position where the clerk standing in front of the checkout counter can easily see it.

Further, in the above-described embodiment, the case where one surveillance camera 4 is arranged in the store ST has been described. The present invention is not limited to this, and an arbitrary number of two or more surveillance cameras 4 may be arranged in the store ST. In that case, the tablet terminal 2 may display the image data received from the plurality of surveillance cameras 4 on the operation display unit 22 while switching at predetermined time intervals. Further, a plurality of tablet terminals 2 may be provided.

Further, in the above-described embodiment, the case where the server 6 is arranged inside the store ST has been described. The present invention is not limited to this, and the server 6 may be arranged outside the store ST. In that case, the server 6 may collectively collect information on the tablet terminals 2 at a plurality of locations.

Further, in the above-described embodiment, the case where the server 6 and the tablet terminal 2 are connected by a wireless LAN has been described. The present invention is not limited to this, and the server 6 and the tablet terminal 2 may be connected by various communication means such as a wired LAN cable.

Further, in the above-described embodiment, the case where the surveillance camera 4 and the tablet terminal 2 are connected by a LAN cable has been described. The present invention is not limited to this, and the surveillance camera 4 and the tablet terminal 2 may be connected by various communication means such as a USB cable and a wireless LAN.

Further, in the above-described embodiment, the inventory monitoring system 1 as an inventory monitoring system is configured by the surveillance camera 4 as an imaging means, the analysis means 40 as an analysis means, and the notification means 42 as a notification means. Said. However, the present invention is not limited to this, and an inventory monitoring system may be configured by an imaging means having various other configurations, an analysis means, and a notification means.

The present invention can also be used in various systems that perform analysis based on image data obtained by capturing images of various products that can be photographed by a camera.

1 …… Inventory monitoring system, 2 …… Tablet terminal, 4 …… Surveillance camera, 6 …… Server, 8 …… Wireless antenna, 10 …… Oden display department, 10S …… Oden sales department, 10A …… Oden exhibition sale Case, 10B ... Loading area, 10T ... Table, 10H ... Heat insulation device, 10C ... Lid, 10HS ... Heat blocking panel, 10U ... USB cable, 12 ... Lunch box display, 20 ... Control , 22 ... Operation display unit, 24 ... Storage unit, 26 ... Camera communication unit, 28 ... Server communication unit, 40 ... Analysis means, 42 ... Notification means, 50 ... Replenishment quantity notification unit, 52 ... … Replenishment notification unit, 54 …… Loaded quantity satisfaction notification unit, ST …… Store, DIP …… Display product imaging screen, IGE …… Egg shape image, LB …… Lunch box, CL …… Container, TDC …… Two-dimensional code , BC …… Bar code, TG …… RFID tag, TB1 …… Egg daily sales quantity table, TB2 …… Egg hour sales quantity table, PD …… Forecast column.

Claims (11)

An imaging means that captures the monitored object and
An analysis means that analyzes an image captured by the image pickup means to determine the loading state of the monitoring target, and an analysis means.
An inventory monitoring system having a notification means for notifying a user of the number of items to be replenished according to the determination result of the loading state. The inventory monitoring system according to claim 1, wherein the notification means displays the captured image and superimposes and displays the number of images to be monitored to be replenished on the captured image. The inventory monitoring system according to claim 2, wherein the notification means clearly indicates the position of the monitoring target to be replenished in the captured image and displays the number of the monitoring targets to be replenished. The inventory monitoring system according to claim 3, wherein the notification means displays the number of monitoring targets to be replenished at the position of the monitoring target to be replenished in the captured image. The inventory monitoring system according to claim 1, wherein the notification means notifies the user of the number of items to be replenished to be monitored without the operation of the user. An imaging means that captures the monitored object and
An analysis means that analyzes an image captured by the image pickup means to determine the loading state of the monitoring target, and an analysis means.
Depending on the determination result of the loading state and the preparation time required until the product to be monitored can be provided, the monitoring target should be replenished in advance by the preparation time from the time to be provided. As a notification means to notify the user that there is
Inventory monitoring system. The inventory monitoring system according to claim 6 , wherein the notification means immediately notifies the user when the preparation time required until the product to be monitored can be provided is not required. The inventory monitoring system according to claim 1 or 6 , wherein the monitoring target is a product placed in a store. The inventory monitoring system according to claim 8 , wherein the monitoring target is a product that is separately arranged for each type of product in a store. The inventory monitoring system according to claim 1 or 6 , wherein the analysis means and the notification means are arranged in an information processing terminal. The analysis means is placed on the server and
The inventory monitoring system according to claim 1 or 6 , wherein the notification means is arranged in an information processing terminal.

Images