JP2022039073A - Information processing apparatus, program, information processing system and information processing method - Google Patents

Abstract

To provide an information processing apparatus, a program, an information processing system and an information processing method that can appropriately manage a quantity of commodities existing in a selling space.SOLUTION: An information processing apparatus (deficiency prevention apparatus 100) comprises: a commodity measurement unit that measures a quantity of a commodity group that is a group of commodities arranged in a selling space included in a quantity estimation unit 113; and the quantity estimation unit 113 that estimates a quantity of the commodities from the quantity of the commodity group and a quantity of each commodity.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, a program, an information processing system, and an information processing method for managing the amount of products existing in the sales floor.

From the viewpoint of avoiding lost opportunities for product sales, it is important to manage the quantity of products in the sales floor. The information processing system described in Patent Document 1 detects a product shortage based on an image of a product shelf, and transports the product from the back room to the store (sales floor). The information processing apparatus described in Patent Document 2 is based on the inventory status of products, such as a time zone other than the time zone when there are many visitors, the time zone before and after the event is held, and the time zone other than the time zone when rainfall is expected. Set the timing of product replenishment in.

Japanese Unexamined Patent Publication No. 2018-147138 Japanese Unexamined Patent Publication No. 2019-149060

The information processing system described in Patent Document 1 replenishes products after a shortage occurs, resulting in a loss of opportunity for product sales. In particular, if there are no products in the back room, the products will be out of stock for a long time, resulting in a large loss. In the information processing apparatus described in Patent Document 2, when the product is in a shortage of inventory, the timing of product replenishment is set to a time zone in which the number of shoppers is expected to be small. However, the timing is not set based on the ever-changing situation such as the number of visitors. Therefore, if the timing of product replenishment is late, shortages may occur.

The present invention has been made in view of such a background, and an object of the present invention is to provide an information processing device, a program, an information processing system, and an information processing method capable of appropriately managing the amount of products existing in the sales floor. ..

In order to solve the above-mentioned problems, the information processing apparatus according to the present invention includes a product measuring unit that measures the quantity of a product group that is a group of products placed in the sales floor, the quantity of the product group, and each product. It is characterized by including a quantity estimation unit that estimates the quantity of the product group from the quantity of.

Further, the program according to the present invention is a program for making a computer function as the information processing apparatus described above.

The information processing system according to the present invention includes a sensor that acquires information on a product group that is a group of products placed in a sales floor, a product measurement unit that acquires the quantity of the product group from the information on the product group, and the product. It is characterized by including a quantity estimation unit that estimates the quantity of the product group from the quantity of the group and the quantity per product.

Further, the information processing method according to the present invention is an information processing method of an information processing apparatus, wherein the information processing apparatus measures a quantity of a product group which is a group of products placed in a sales floor, and the product. It is an information processing method characterized by executing a step of estimating the quantity of the product group from the quantity of the group and the quantity per product.

According to the present invention, it is possible to provide an information processing device, a program, an information processing system and an information processing method capable of appropriately managing the amount of products existing in the sales floor.
Issues, configurations and effects other than those described above will be clarified by the description of the following embodiments.

It is an overall block diagram of the shortage prevention system including the shortage prevention device which concerns on this embodiment. It is a functional block diagram of the shortage prevention device which concerns on this embodiment. It is a data structure diagram of the store visitor number database which concerns on this embodiment. It is a data structure diagram of the daily store visitor number database which concerns on this embodiment. It is a data structure diagram of the product database which concerns on this embodiment. It is a data composition diagram of the quantity forecast database which concerns on this embodiment. It is a block diagram of the replenishment required product page of the Web server which concerns on this embodiment. It is a flowchart of the store visitor analysis processing which concerns on this embodiment. It is a flowchart of the product registration process which concerns on this embodiment. It is a flowchart of the quantity prediction processing which concerns on this embodiment. It is a data structure diagram of the store visitor database which concerns on the modification of this embodiment. It is a data structure diagram of the store stay database which concerns on the modification of this embodiment. It is a flowchart of the store visitor analysis process which concerns on the modification of this embodiment.

≪Overall configuration of shortage prevention system≫
Hereinafter, a shortage prevention system including a shortage prevention device (information processing device) in the embodiment (embodiment) for carrying out the present invention will be described. The shortage prevention device measures the quantity (product quantity) of products from the bulk, area, quantity, weight, etc. of the products in the sales floor using a camera, range sensor, three-dimensional sensor, scale (weighing scale), etc. (presume. In addition, the shortage prevention device predicts the sales volume from past sales results, attributes of shoppers (gender and generation (age)), weather, time zone, events in the commercial area, and products that are in the sales floor after a predetermined time. Predict the quantity of. The shortage prevention device issues (notifies) an alarm if the predicted product quantity is less than or equal to a predetermined value. Upon receiving the alarm, the clerk replenishes the product or the supplier of the product ships the product.

In this way, by issuing an alarm in advance when the quantity of products is expected to fall below the specified value after a predetermined time, the clerk or the supplier can replenish or ship the products, and the product is out of stock. Can be avoided. In addition, since an alarm is received before a shortage occurs, the store clerk or supplier can replenish or ship the product at an appropriate timing such as when the number of visitors is small or when it is convenient for him / herself. become able to.

≪Overall configuration of shortage prevention device≫
FIG. 1 is an overall configuration diagram of a shortage prevention system 10 including a shortage prevention device 100 according to the present embodiment. The shortage prevention system 10 includes a shortage prevention device 100, an entrance / exit camera 210, a sales floor camera 220, a distance measuring sensor 230, a three-dimensional sensor 240, a registration camera 250, a display panel 260, a warning light 270, and a Web server 810. Consists of.
The doorway camera 210 is a camera installed at the doorway of a store, and photographs a person entering or leaving the store. Using image recognition technology (person recognition technology, face recognition technology) from the images taken by the store occupants, the number of store occupants by gender and age (generation) can be measured.

The sales floor camera 220 is a camera installed in the sales floor of the product and photographs the product. One sales floor camera 220 may be installed so as to be able to shoot an entire product, or a plurality of sales floor cameras 220 may be installed so as to be able to shoot an entire product. The 220 may be installed so that the entire plurality of products can be photographed. The quantity of products can be counted from the image of the product using image recognition technology (object detection technology). Alternatively, the product quantity can be counted from the area of the entire product arranged in one row and the area of one registered product.

The distance measuring sensor 230 measures the distance to the product. For example, if the product is a vegetable and is piled up, the volume of the piled up product is estimated by estimating the bottom area of the mountain from the image taken by the sales floor camera 220 and measuring the height of the mountain with the distance measuring sensor 230. (Size, volume) can be calculated. Further, the product quantity can be calculated from the bulk of one registered product.
The three-dimensional sensor 240 acquires the distance (depth) to the product as image data. By measuring the pile of products from above the piled products with the three-dimensional sensor 240, the bulk of the pile of products can be calculated, and the quantity of products can be calculated. Further, the bulk of the product can be calculated by measuring the product from above the flatly stacked products with the three-dimensional sensor 240, and the quantity of the product can be calculated.

The registration camera 250 is a camera used when registering a product in the shortage prevention device 100. When the product is photographed by the registration camera 250, the shortage prevention device 100 measures the bulk of the product and the area (dimensions) of the product from the photographed image, and registers the product in association with the product. The registered bulk and area (dimensions) are used when calculating the product quantity.

The display panel 260 displays information on products whose quantity is expected to fall below a predetermined value after a predetermined time, and issues an alarm.
The warning light 270 lights up when there is a product whose quantity is expected to fall below a predetermined value after a predetermined time, and issues an alarm.
The Web server 810 posts products whose quantity is expected to fall below a predetermined value after a predetermined time.

≪Functional configuration of shortage prevention device≫
FIG. 2 is a functional configuration diagram of the shortage prevention device 100 according to the present embodiment. The shortage prevention device 100 includes a control unit 110, a storage unit 120, and an input / output unit 109. The input / output unit 109 transmits / receives data to / from the entrance / exit camera 210, the sales floor camera 220, the distance measuring sensor 230, the three-dimensional sensor 240, the registration camera 250, the display panel 260, the warning light 270, and the Web server 810. .. The input / output unit 109 also transmits / receives data to / from a display (not shown), a keyboard, and a mouse (not shown).

The storage unit 120 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), etc. ), Daily store occupants database 140 (described as daily store occupants DB in FIG. 2), product database 150 (described as product DB in FIG. 2), quantity forecast database 170 (quantity forecast DB in FIG. 2). Description), and program 121 is stored. The program 121 includes a description of the processing procedure of FIGS. 8 to 10 described later.

≪Functional configuration of shortage prevention device: database of number of visitors≫
FIG. 3 is a data structure diagram of the number of visitors database 130 according to the present embodiment. The number of store visitors database 130 stores the number of store visitors by attribute (gender / generation) at predetermined intervals within a predetermined time. The store occupant number database 130 shown in FIG. 3 stores the number of store occupants by gender and generation at 5-minute intervals within the last 30 minutes. The generations are divided into those in their 20s, 20s / 30s, 40s / 50s, and 60s or older. In addition, the number of visitors of unknown gender or generation is stored in "Unknown".

FIG. 4 is a data structure diagram of the daily number of store visitors database 140 according to the present embodiment. The daily number of store visitors database 140 stores the number of daily store visitors by attribute. The classification of the daily number of store entrants database 140 by attribute is the same as that of the store occupant number database 130.

≪Functional configuration of shortage prevention device: Product database≫
FIG. 5 is a data structure diagram of the product database 150 according to the present embodiment. The product database 150 is a database for storing information related to products, and is, for example, tabular data. A row (record) of the product database 150 indicates one product and is composed of a name 151, a category 152, a camera 153, a dimension 154, a bulk 155, an image 156, a registrant 157, and a contact 158.

The name 151 is the name of the product. Category 152 is a product category to which the product belongs. The camera 153 is identification information of the sales floor camera 220 that captures the sales floor of the product.
The dimension 154 is the size of the product, which is the size of the product (area of the product) seen from the sales floor camera 220 identified by the camera 153. The dimension 154 may be the length and width of the product, the area of the product, or the number of pixels of the product in the image taken by the sales floor camera 220 identified by the camera 153. It may be the total number of pixels of the product in the image taken by the sales floor camera 220 identified by the camera 153.

The bulk 155 is the bulk (volume, size) of the product. Image 156 is an image of a product. The registrant 157 is the name of the clerk who registered the product. Contact 158 is the contact of the clerk or supplier who replenishes the goods.
Regarding the dimensions 154 and bulk 155, the product registration unit 112, which will be described later, measures (estimates) the dimensions and volume of the product from the captured image of the registration camera 250, and the numerical values of the estimation results are registered.

≪Functional configuration of shortage prevention device: Quantity forecast database≫
FIG. 6 is a data structure diagram of the quantity forecast database 170 according to the present embodiment. The quantity forecast database 170 is a database for storing information related to the quantity forecast of goods, and is, for example, tabular data. The rows (records) of the quantity forecast database 170 relate to one product, name 171, quantity 172, replenishment threshold 173, base reduction rate 174, real reduction rate 175, 30-minute forecast quantity 176, 60-minute forecast quantity 177. , Prediction history 178, Attribute coefficient 179, Weather coefficient 180, Temperature coefficient 181, Morning coefficient 182, Afternoon coefficient 183, Evening coefficient 184, Statistical coefficient 185, Event coefficient 186, Last replenishment time 187, and Alarm 188 columns (attributes) )including.

The name 171 is the name of the product. The quantity 172 is the quantity of goods in the sales floor.
The replenishment threshold value 173 is a threshold value for replenishing the product. If the predicted quantity after 30 minutes, which will be described later, or the predicted quantity after 60 minutes is 173 or less of the replenishment threshold value, an alarm for product replenishment is issued (notified).
The base decrease rate 174 is the product sales amount (decrease amount of the product quantity) per unit time (for example, one hour) set from the past sales results. The real decrease rate 175 is the amount of merchandise sold per unit time in which the base decrease rate 174 is added to the number of customers, the weather, the time zone, and the like. In the following description, the unit time will be described as one hour.

The predicted quantity after 30 minutes 176 is a predicted value of the product quantity after 30 minutes predicted from the quantity 172 and the real decrease rate 175. The predicted quantity 177 after 60 minutes is a predicted value of the product quantity after 60 minutes predicted from the quantity 172 and the real decrease rate 175. The forecast history 178 is a history of the forecast quantity 176 after 30 minutes and the forecast quantity 177 after 60 minutes. The shortage prevention device 100 stores the prediction result in the predicted quantity 176 after 30 minutes and the predicted quantity 177 after 60 minutes for each prediction, and the old predicted quantities 176 and 60 after 30 minutes before the prediction result is stored. The predicted quantity after minutes 177 and the predicted quantity 177 are stored in the prediction history 178.

The attribute-specific coefficient 179 is a coefficient used when calculating the real reduction rate 175 based on the number of store entrants by attribute (see the store occupant number database 130 shown in FIG. 3). The attribute-specific coefficient 179 includes a gender / generation-specific coefficient included in the store occupant number database 130. The attribute-specific coefficient 179 is, for example, a value of "1.2, 1.2, 1.1, 1.0, ...", And the first "1.2" is a coefficient for women in their twenties or younger. The next "1.2" is the coefficient for women in their 20s and 30s. Below, the coefficient for women in their 40s and 50s, the coefficient for women in their 60s and above, the coefficient for men under 20s, the coefficient for men in their 20s and 30s, the coefficient for men in their 40s and 50s, the coefficient for men in their 60s and above, Followed by an unknown coefficient.

The coefficient is a coefficient to be multiplied when the real decrease rate 175 is calculated from the base decrease rate 174, and is a numerical value to be multiplied when the number of visitors in the last 10 minutes exceeds, for example, 10 people. For example, assume that the coefficient for women under the age of 20 is "1.2". When the number of female customers under the age of 20 in the last 10 minutes exceeds 10, the real decrease rate 175 is calculated by multiplying the base decrease rate 174 by 1.2. Furthermore, when the number of female customers in their 20s and 30s exceeds 10 in the last 10 minutes, the base decrease rate 174 is further multiplied by 1.2 to calculate the real decrease rate 175.

The weather coefficient 180 is a coefficient used when calculating the real reduction rate 175 based on the weather. The weather coefficient 180 includes a coefficient for each weather. The weather coefficient 179 is, for example, a value of "1, 1, 1.2", and indicates a coefficient in the case of sunny weather, cloudy weather, and rainy weather, respectively. A real reduction rate of 175 is calculated by multiplying by a coefficient according to the weather at the time of prediction.
The air temperature coefficient 181 is a coefficient used when calculating the real reduction rate 175 based on the air temperature. The temperature coefficient 181 includes a coefficient for each of the high and low temperatures of the normal or the previous day. The weather coefficient 179 is, for example, a value of "1,1,0.9", and indicates a coefficient when the temperature is higher, the same, or lower than that of the normal year or the previous day, respectively. A real reduction rate of 175 is calculated by multiplying by a coefficient according to the weather at the time of prediction.

The morning coefficient 182 is a coefficient used when calculating the real reduction rate 175 when the predicted time is in the morning. The afternoon coefficient 183 is a coefficient used when calculating the real reduction rate 175 when the predicted time is afternoon. The evening coefficient 184 is a coefficient used when calculating the real reduction rate 175 especially in the evening (for example, after 6:00 pm or after sunset) even when the predicted time is in the afternoon.
The statistical coefficient 185 is a coefficient set by comparing the predicted value at the time of prediction included in the prediction history with the quantity 172, and is a coefficient used when calculating the real reduction rate 175. For example, if the quantity 172 is larger than the predetermined quantity or the predetermined ratio or more than the predicted quantity 176 after 30 minutes predicted 30 minutes before, it is determined that the predicted quantity 176 after 30 minutes is too small (the sales forecast quantity is excessive). The coefficient is set to be less than 1.

The event coefficient 186 is a coefficient set according to an event in the vicinity of the store (trade area). For example, if the event is an elementary school athletic meet, a large coefficient is set at the time of predicting the time zone before and after the athletic meet.
The final replenishment time 187 is the final time when the product can be replenished. For example, it is a time when a delivery company can be requested to replenish and deliver the product within the same day.
The alarm 188 is a flag indicating whether or not the alarm (replenishment notification) is issued (“Yes”) or not (“No”) when the predicted quantity after 30 minutes or the predicted quantity 177 after 60 minutes becomes the replenishment threshold value 173 or less. ..

≪Functional configuration of shortage prevention device: Control unit≫
Returning to FIG. 2, the control unit 110 includes a CPU (Central Processing Unit). The control unit 110 includes a person analysis unit 111, a product registration unit 112, a quantity estimation unit 113, and a quantity prediction unit 114.

≪Functional configuration of shortage prevention device: Person analysis department≫
The person analysis unit 111 analyzes the number of visitors by attribute (gender / generation) from the image taken by the entrance / exit camera 210 and stores it in the number of visitors database 130 (see FIG. 3). The person analysis unit 111 further updates the daily store visitor number database 140. The store entrant analysis process executed by the person analysis unit 111 will be described with reference to FIG. 8 described later.

≪Functional configuration of shortage prevention device: Product registration department≫
The product registration unit 112 registers product information in the product database 150 from images taken by the registration camera 250. When registering a product, the registrant of the product faces the registration camera 250 with a medium (for example, a board or paper) in which basic information of the product such as the product name, category, and identification information of the sales floor camera 220 is described. ) And then the product. The product registration unit 112 acquires the registration contents from the presented medium or product. The product registration unit 112 may acquire the registration content from the information input by the registrant using a keyboard (not shown) connected to the input / output unit 109 (see FIG. 2).

Specifically, the product registration unit 112 acquires the product name and category, the identification information of the sales floor camera 220, the registrant, the contact information, and the dimensions and bulk of the product from the image taken by the registration camera 250. The product name and category, the identification information of the sales floor camera 220, the registrant, and the contact information may be obtained, for example, the information described as characters on the board by using character recognition technology, or may be described on the product package. It may be read from code information such as QR code (registered trademark) or barcode, or character information.

As for the category, the category may be acquired from the product name by referring to the database for associating the product name with the category in or outside the shortage prevention device 100. For the identification information of the sales floor camera 220, the identification information of the sales floor camera 220 is acquired from the product name by referring to the database that associates the product name in or outside the shortage prevention device 100 with the identification information of the sales floor camera 220. You may.

Regarding the image of the product, the product registration unit 112 cuts out an area in which the product is reflected from the image taken by the registration camera 250 and acquires it.
The product registration unit 112 may identify and acquire the registrant from the face image reflected on the registration camera 250 by using the face recognition technique.

As for the contact information, the product registration unit 112 may obtain the contact information from the registrant by referring to the database for associating the registrant with the contact information in the shortage prevention device 100 or outside. Contact information may be identified using facial recognition technology from the producer's face photo on the board or product package. The contact information may be specified from the information in the box containing the product (for example, the producer information in the form of characters or QR code).
The product registration unit 112 uses the acquired name, category, identification information of the sales floor camera, product image, registrant, and contact information as the name 151, category 152, camera 153, and image 156 of the product database 150 (see FIG. 5). Register with registrant 157 and contact information 158, respectively.

When registering a product, the registrant registers the surface (orientation) of the product taken by the sales floor camera 220 toward the registration camera 250 longer than the other surfaces. For example, if the products are lined up on a table instead of a shelf, the top side of the products should face the registration camera 250 longer than the sides. The product registration unit 112 registers the dimension of the surface of the product taken for a long time as the dimension 154, calculates the bulk of the product from the dimension 154 and the length (height) of the side surface, and registers it in the bulk 155.

For products such as vegetables and fruits that are piled up in the sales floor, the registrant registers a plurality of surfaces such as the top surface and the side surface of the product toward the registration camera 250. The product registration unit 112 measures the dimensions and bulk of the product from the captured images of the top surface and the side surface of the product. As a measurement method, dimensions and bulk may be measured from the vertical and horizontal lengths and shapes of the product, or measurement may be performed using machine learning technology that estimates dimensions and bulk from images of multiple surfaces such as the top and side surfaces of the product. It may be (estimated).

≪Functional configuration of shortage prevention device: Quantity estimation unit≫
The quantity estimation unit 113 estimates the quantity of the goods in the sales floor and stores it in the quantity 172 of the quantity prediction database 170 (see FIG. 6). Specifically, the quantity estimation unit 113 estimates the product quantity from the image taken by the sales floor camera 220. For example, when products are lined up in the sales floor, the quantity of products is estimated using image analysis technology (object recognition technology). When the products are arranged in one row on the table, the area of the image showing the products may be divided by the dimension 154 (see FIG. 5) to obtain the quantity. Alternatively, the quantity may be calculated by finding the area of the blank where there is no product in the table and subtracting the area of the blank from the area of the table to obtain the area where the product is reflected. When the products are lined up on the shelves, the image of the sales floor camera 220 and the measurement result of the distance measuring sensor 230 are combined to obtain the size / volume of the empty space on the shelves, and the quantity of the products in the sales floor is calculated. It may be calculated. The volume of the empty space on the shelf measured by the three-dimensional sensor 240 may be obtained to calculate the quantity of products in the sales floor.

When the products are piled up, the quantity estimation unit 113 combines the image of the sales floor camera 220 and the measurement result of the distance measuring sensor 230 to obtain the bulk of the pile, and divides it by the bulk 155 (see FIG. 5). You may ask for the quantity. In the case of piles, there are gaps in the products, and it is considered that the value obtained by simply dividing the bulk of the pile by the bulk 155 (see FIG. 5) is larger than the actual quantity. In consideration of this point, it may be divided by a value larger than the bulk 155. The same applies to the case where the quantity is obtained from the bulk of the mountain measured by the three-dimensional sensor 240.

≪Functional configuration of shortage prevention device: Quantity prediction unit≫
The quantity prediction unit 114 calculates the forecast quantity after 30 minutes (estimated quantity after 30 minutes 176) and the forecast quantity after 60 minutes (estimated quantity after 60 minutes) from the quantity 172 of the current product (see FIG. 6). .. Further, the quantity prediction unit 114 issues an alarm to replenish if the calculation result is the replenishment threshold value 173 or less.
Specifically, the quantity prediction unit 114 obtains the real reduction rate 175 from the base reduction rate 174, and obtains the predicted quantity 176 after 30 minutes and the predicted quantity 177 after 60 minutes. When the quantity prediction unit 114 obtains the real decrease rate 175, if the number of visitors by gender / generation in the last 10 minutes is 10 or more in the base decrease rate 174 and the attribute-specific coefficient 179, the relevant person is concerned. Multiply the coefficients by gender and generation. When the number of visitors by a plurality of genders / generations is 10 or more, the quantity prediction unit 114 multiplies each coefficient.

The quantity prediction unit 114 multiplies the result of multiplying the attribute-specific coefficient 179 by the coefficient in the weather coefficient 180 according to the weather at the time of prediction, and further multiplies the coefficient in the temperature coefficient 181 according to the temperature at the time of prediction. .. The quantity prediction unit 114 multiplies the result of multiplying the temperature coefficient 181 by any one of the morning coefficient 182, the afternoon coefficient 183, and the evening coefficient 184 according to the time at the time of prediction. The quantity prediction unit 114 further stores the result of multiplying the statistical coefficient 185 and the event coefficient 186 in the real reduction rate 175.

Next, the quantity prediction unit 114 sets the quantity 172 indicating the current quantity of goods and the real decrease rate 175 indicating the number of decreases in the quantity per hour to the predicted quantity 176 after 30 minutes and the predicted quantity 177 after 60 minutes. Is calculated. Specifically, the quantity prediction unit 114 calculates the predicted quantity 176 after 30 minutes by subtracting 1/2 of the real decrease rate 175 from the quantity 172, and subtracts the real decrease rate 175 from the quantity 172 for the predicted quantity 177 after 60 minutes. calculate. The quantity prediction unit 114 adds the stored 30-minute forecast quantity 176 and the 60-minute forecast quantity 177 to the forecast history 178, and sets the calculated values as the 30-minute forecast quantity 176 and the 60-minute forecast quantity 177. Store in.

Subsequently, the quantity prediction unit 114 issues an alarm if the predicted quantity 176 after 30 minutes or the predicted quantity 177 after 60 minutes is the replenishment threshold value 173 or less and before the final replenishment time 187. The quantity prediction unit 114 displays the time and the product name to be alarmed on the display panel 260 (see FIG. 1). Further, if the 30-minute predicted quantity 176 is equal to or less than the replenishment threshold value 173, the quantity prediction unit 114 displays the 30-minute predicted quantity 176 as the predicted quantity after 30 minutes and the 60-minute predicted quantity 177 on the display panel 260. If the replenishment threshold is 173 or less, the predicted quantity after 60 minutes is displayed as the predicted quantity after 60 minutes. The quantity prediction unit 114 may turn on the warning light 270 and prompt the user to look at the display panel 260.

Alternatively, the quantity prediction unit 114 may send a notification of the content to be displayed on the display panel 260 described above to the contact information 158 (see FIG. 5) of the product subject to the alarm to issue an alarm.
Further, the quantity prediction unit 114 may upload an image or a video of the sales floor camera 220 that captures the alarm target product to the Web server 810. When the contact information 158 is not a clerk but a supplier, the actual sales floor status can be confirmed by receiving the notification and referring to the Web server 810.

FIG. 7 is a configuration diagram of the replenishment-required product page 811 of the Web server 810 according to the present embodiment. The image of the sales floor of the lunch box A is displayed on the left side of the product page 811 requiring replenishment, and the image of the sales floor of the lunch box B is displayed on the right side. By referring to this page, the supplier can grasp the status of the products he / she is delivering even if he / she is away from the store.

≪Store visitor analysis processing≫
FIG. 8 is a flowchart of the store visitor analysis process according to the present embodiment. The store clerk analysis process is a process that is repeatedly executed every 5 minutes. A process of calculating the number of store entrants by gender and generation, which is executed by the person analysis unit 111, will be described with reference to FIG.
In step S11, the person analysis unit 111 repeats the processes of steps S12 and S13 for 5 minutes.

In step S12, the person analysis unit 111 analyzes the image of the entrance / exit camera 210 to detect the entrant.
In step S13, the person analysis unit 111 estimates the gender and generation of the person detected in step S12. The generation may be under 20s, 20s / 30s, 40s / 50s, 60s or older. If the gender or generation is unknown, it is unknown.
In step S14, the person analysis unit 111 calculates the cumulative value of the number of visitors by gender and generation detected in 5 minutes. If it is unknown, the cumulative value is calculated as unknown.

In step S15, the person analysis unit 111 updates the store visitor number database 130 (see FIG. 3). Specifically, the person analysis unit 111 adds a new record (row) to the store visitor number database 130, stores the start time of the store visitor analysis process in the "time zone" of the record, and other genders. -The cumulative value calculated in step S14 is stored in the column for each generation. Next, the person analysis unit 111 deletes the record of the number of visitors database 130 having the oldest "time zone".
In step S16, the person analysis unit 111 updates the daily number of store entrants database 140 (see FIG. 4). Specifically, the person analysis unit 111 adds the cumulative value calculated in step S14 to the column for each gender / generation in the record corresponding to the day when "day / day" starts the store visitor analysis process. If there is no record in the daily number of store entrants database 140 that matches the date when "day and day" started the store clerk analysis process, a new record is added and entered in the "day and day" column. The date when the shopkeeper analysis process is started is stored, and the cumulative value calculated in step S14 is stored in the other columns (columns) for each gender / generation.

≪Product registration process≫
FIG. 9 is a flowchart of the product registration process according to the present embodiment. The product registration process is a process executed when a product is registered. When registering a product, the registrant of the product faces the registration camera 250 with a medium (for example, board or paper) in which basic information of the product such as the product name, category, and identification information of the sales floor camera is described. And then present the product.

In step S21, the product registration unit 112 acquires basic product information and registrant information from the image of the registration camera 250. Specifically, the product registration unit 112 acquires the product name, category, and identification information of the sales floor camera from the video of the presented medium. Further, the product registration unit 112 detects the registrant from the image of the registration camera 250 and acquires the name and contact information of the registrant.
In step S22, the product registration unit 112 detects the product from the image of the registration camera 250. Subsequently, the product registration unit 112 cuts out an image of the detected product from the video and acquires it.
In step S23, the product registration unit 112 measures the dimensions and bulk of the product detected in step S22.

In step S24, the product registration unit 112 stores the name, category, identification information of the sales floor camera, dimensions, bulk, image, registrant, and contact information obtained in steps S21 to S23 in the product database 150 (see FIG. 5). .. Specifically, if the product registration unit 112 has a record of the product database 150 having the same name 151 as the name obtained in step S21, the name, category, identification information and dimensions of the sales floor camera 220 obtained in steps S21 to S23 , Bulk, product image, registrant, and contact information are stored in the record name 151, category 152, camera 153, dimension 154, bulk 155, image 156, registrant 157, and contact information 158, respectively. If there is no record of the product database 150 having the same name 151 as the name obtained in step S21, the product registration unit 112 obtains the name, category, identification information, dimensions, and bulk of the sales floor camera 220 obtained in steps S21 to S23. The product image, registrant, and contact information are stored in the newly added record name 151, category 152, camera 153, dimension 154, bulk 155, image 156, registrant 157, and contact information 158, respectively.

≪Quantity forecast processing≫
FIG. 10 is a flowchart of the quantity prediction process according to the present embodiment. The quantity prediction process is performed at a predetermined timing for each product, for example, the timing after the number of visitors database 130 is updated, the replenishment threshold 173 of the quantity prediction database 170 (see FIG. 6), the base reduction rate 174, and the coefficient for each attribute. At the timing after any one of 179, weather coefficient 180, temperature coefficient 181, morning coefficient 182, afternoon coefficient 183, evening coefficient 184, and event coefficient 186 is set, and at the timing after the quantity 172 changes by a predetermined value or more. Will be executed. The quantity forecasting process may be executed periodically for each product.

In step S31, the quantity estimation unit 113 estimates the quantity of the goods in the sales floor.
In step S32, the quantity estimation unit 113 stores the quantity estimated in step S31 in the quantity 172 of the quantity prediction database 170 (see FIG. 6).
In step S33, the quantity prediction unit 114 has a base decrease rate of 174, an attribute-specific coefficient of 179, a weather coefficient of 180, and a temperature coefficient of 181 according to the attributes and number of shoppers, weather, temperature, time zone, events in the commercial area, and the like. , Morning coefficient 182, Afternoon coefficient 183, Evening coefficient 184, Statistical coefficient 185, and Event coefficient 186 to calculate and store the real reduction rate 175.

In step S34, the quantity prediction unit 114 calculates and updates the quantity 176 predicted after 30 minutes and the predicted quantity 177 after 60 minutes from the quantity 172 and the real decrease rate 175. Before updating the predicted quantity 176 after 30 minutes and the predicted quantity 177 after 60 minutes, the value before the update is stored in the prediction history 178 as a history.
In step S35, the quantity prediction unit 114 proceeds to step S36 if the predicted quantity 176 after 30 minutes or the predicted quantity 177 after 60 minutes is the replenishment threshold value 173 or less (step S35 → YES). If the predicted quantity 176 after 30 minutes and the predicted quantity 177 after 60 minutes are larger than the replenishment threshold value 173 (step S35 → NO), the quantity prediction unit 114 proceeds to step S38.

In step S36, the quantity prediction unit 114 proceeds to step S37 if the current time is before the final replenishment time 187 (step S36 → YES), and proceeds to step S39 if the current time is after the final replenishment time 187 (step S36 → NO). The final replenishment time 187 is the final time when the product can be replenished.
In step S37, the quantity prediction unit 114 issues an alarm if the alarm 188 is “absent” and updates the alarm 188 to “yes”. Specifically, the quantity prediction unit 114 displays an alarm (notification) including the product name and the predicted quantity on the display panel 260, notifies the contact information 158 (see FIG. 5), and displays the image of the sales floor on the Web server 810 (see FIG. 5). Image) is uploaded. If the alarm 188 is "Yes", the alarm may or may not be issued again.

In step S38, the quantity prediction unit 114 updates the alarm 188 to “no” if it is “yes”.
In step S39, the quantity prediction unit 114 corrects the statistical coefficient 185. Specifically, if the quantity 172 is larger than the predetermined quantity or the predetermined ratio with respect to the predicted quantity 176 after 30 minutes predicted 30 minutes before stored in the forecast history 178, the predicted quantity 176 after 30 minutes is too small (sales). It is determined that the expected amount is excessive), and the statistical coefficient 185 is set to be smaller than 1. Further, if the quantity 172 is smaller than the predetermined quantity or the predetermined ratio with respect to the predicted quantity 176 after 30 minutes predicted 30 minutes before stored in the forecast history 178, the predicted quantity 176 after 30 minutes is excessive (sales forecast). It is determined that the amount is too small), and the statistical coefficient 185 is set to be larger than 1. The quantity prediction unit 114 may be compared with the 60-minute predicted quantity 177 stored in the prediction history 178 before 60 minutes, or the 30-minute predicted quantities 176 and 60 predicted 30 minutes before. It may be compared with both the predicted quantity 177 after 60 minutes predicted before the minute.

≪Characteristics of shortage prevention system≫
The above-mentioned shortage prevention system 10 (out-of-stock prevention device 100) predicts the quantity of products in the sales floor from images taken by the sales floor camera 220 and measurement results of the three-dimensional sensor 240. In addition to the base reduction rate of 174 set from the sales results of past products, the shortage prevention device 100 can be used to check the status of shoppers (see coefficient 179 by attribute), weather, temperature, time zone, events in the trade area, etc. A real decrease rate of 175 of the product quantity is calculated in consideration of this, and the number of products after a predetermined time is predicted. If the predicted number of products is equal to or less than a predetermined value (replenishment threshold 173), an alarm is issued.

By introducing the shortage prevention system 10, an alarm is issued when the number of products is small, and the clerk can replenish the products. As a result, it becomes possible to reduce shortages and reduce opportunity loss of product sales. It is not necessary for the clerk to look around the store and check the quantity of products, which can reduce the work load of the clerk.

In addition, by notifying the supplier of the alarm, it is possible to reduce the ordering work of the clerk. The supplier is notified when the quantity of the goods is reduced and the shortage is predicted, so that the supplier can deliver the goods at his / her convenient timing before the shortage occurs. It is also possible to browse the Web server 810 and check the status of the product.
By browsing the Web server 810, the customer of the store can know the product that may be out of stock, and can go to the store and purchase the product before the product is out of stock.

≪Variation example: Store entrant analysis processing considering store time≫
The above-mentioned store clerk analysis process (see FIG. 8) analyzes only store occupants. By analyzing the number of people who have left the store, the time spent in the store can be measured, and a more accurate real reduction rate of 175 (see FIG. 6) can be calculated.
FIG. 11 is a data structure diagram of the store clerk database 310 according to a modified example of the present embodiment. The store clerk database 310 is, for example, tabular data, and one row (record) indicates information of each store clerk. The record includes attributes of facial features 311, entry time 312, exit time 313, and gender / generation 314.
The facial feature 311 is facial feature information for identifying a visitor. The entry time 312 is the time when the entrant enters the store, and the exit time 313 is the time when the entrant leaves the store. Gender / generation 314 is the estimated gender and generation of the store occupants.

FIG. 12 is a data structure diagram of the store-based time database 320 according to a modified example of the present embodiment. The store-based time database 320 is a database that stores the total store-based hours by gender and generation. For example, the top row of the store-based time database 320 shown in FIG. 12 shows that the total store-based time for women under the age of 20 is 10 minutes during the time period from 10:00 to 10:05, and women in their 20s. It shows that the total time spent in the store in the 30s is 16 minutes.

FIG. 13 is a flowchart of the store visitor analysis process according to the modified example of the present embodiment. The store visitor analysis process shown in FIG. 13 is a process of simultaneously executing the three processes of steps S41 to S44, steps S46 to S48, and steps S51 to S53. Hereinafter, the person analysis unit 111 that executes the store visitor analysis process according to the modified example of the present embodiment will be referred to as the person analysis unit 111A.

In step S41, the person analysis unit 111A repeats the processes of steps S42 to S44 for each visitor.
In step S42, the person analysis unit 111A extracts the facial features of the entrant.
In step S43, the person analysis unit 111A estimates the gender and generation of the entrant.
In step S44, the person analysis unit 111A registers in the store clerk database 310 (see FIG. 11, described as the store clerk DB in FIG. 13). Specifically, the person analysis unit 111A adds a record to the store entrant database 310, displays the face feature extracted in step S42 on the face feature 311 as the current time at the store entry time 312, and "-" at the store exit time 313. , And the gender and generation estimated in step S43 are stored in the gender / generation 314.

In step S46, the person analysis unit 111A repeats the processes of steps S47 to S48 for each person who leaves the store.
In step S47, the person analysis unit 111A extracts the facial features of the person who left the store.
In step S48, the person analysis unit 111A updates the store occupant database 310. Specifically, the person analysis unit 111A collates the store entrant database 310 with the face feature extracted by the face feature 311 in step S47, identifies the record whose exit time 313 is "-", and determines the record. The record is updated to the current time at the closing time 313.

In step S51, the person analysis unit 111A repeats the processes of steps S52 to S53 every 5 minutes.
In step S52, the person analysis unit 111A calculates the store stay time for the last 5 minutes for each gender and generation. For example, assuming that the current time is 10:05, the total time (minutes) of being in the store from 10:00 to 10:05 is calculated by gender and generation with reference to the store clerk database 310. For example, for women under the age of 20, the record that the entry time 312 is 9:56 and the exit time 313 is 10:03, and the entry time 312 is 10: 1 and the exit time 313 is "-". If there is a record, the total time will be 7 minutes.
In step S53, the person analysis unit 111A updates the store time database 320 (see FIG. 12). Specifically, the person analysis unit 111A adds a record to the store presence time database 320, stores the time 5 minutes before in the time zone attribute (column), and stores the total time calculated in step S52 in the other attributes. Is stored by gender and generation.

By referring to the store-based time database 320, the quantity prediction unit 114 can refer to the store-based time by gender and generation. The quantity prediction unit 114 may calculate the real reduction rate 175 (see FIG. 6) by referring to the store stay time instead of the number of visitors. For example, if a woman in her twenties or less has been in the store for the last 10 minutes for more than 40 minutes, the attribute-specific coefficient 179 corresponding to the woman in her twenties may be multiplied. Alternatively, the quantity prediction unit 114 may calculate the real reduction rate 175 by referring to both the store occupant number database 130 and the store occupant database 310. By calculating the real reduction rate 175 from a plurality of viewpoints, it becomes possible to calculate the real reduction rate 175 with higher accuracy.

The number of visitors database 130 (see FIG. 3) stores only the number of visitors per unit time (5 minutes). In this case, the number of visitors and the number of residents in the store are almost the same, but there is a time lag to be exact. be.
By introducing the store clerk database 310 (see FIG. 11), the entrance and exit of individual store occupants can be managed, so it is possible to more accurately calculate how many people are in the store at the relevant time (the store exit time 313 is "-". There are as many people in the store as there are people). Specifically, in the store-based time database 320 (see FIG. 12), the number of people in the store may be stored instead of the total time spent in the store. For example, the number of people in the store at 10:00 is reflected in the store time database 320 with reference to the store clerk database 310. Next, processing is performed so as to reflect the number of people in the store at 10:05. By processing in this way, the numerical value (10, 16, 30 ...) of each attribute of the store-based time database 320 becomes the actual number of store-occupied people, not the store-based time. The coefficient for each attribute will be reflected more accurately, and the real reduction rate 175 (see FIG. 6) with high accuracy can be calculated.

≪Variation example: Product quantity≫
The quantity estimation unit 113 estimates the quantity of the product from the information of the sales floor camera 220, the distance measuring sensor 230, and the three-dimensional sensor 240. On the other hand, when the sales floor (the table on which the product is placed) itself is a scale, the quantity may be estimated from the weight of the product. In the product registration process (see FIG. 9), instead of acquiring the dimensions 154 and the bulk 155 (see FIG. 5) from the image of the registration camera 250, the registrant acquires the weight per product from the scale on which the product is placed. Then, it may be registered in the product database 150. The quantity can be obtained by dividing the weight of the product in the sales floor by the weight registered in the product database 150.

<< Modification example: Replenishment threshold >>
In the above-described embodiment, the quantity of the product is used as the base, and the quantity and the threshold value of the quantity after a predetermined time are set to issue an alarm. For products that are piled up, the bulk of the products may be used instead of the quantity. The replenishment threshold 173, the base reduction rate 174, the real reduction rate 175, the predicted quantity after 30 minutes 176, and the predicted quantity 177 after 60 minutes may be quantified as bulk (volume) instead of quantity.

≪Other variants≫
In the present specification, the steps for describing a program include not only processes performed in chronological order but also processes executed in parallel or individually, even if they are not necessarily processed in chronological order. .. Further, the order of the steps may be changed within the range in which the same effect or function can be obtained.

Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. For example, the final replenishment time 187 (see FIG. 6) and the alarm 188 are provided in the quantity prediction database 170, but may be provided in the product database 150 (see FIG. 5).

The quantity estimation unit 113 calculates the quantity of the product from the quantity of the product (product group) (the bulk of the piled products, the area of the products arranged in one row, the weight of the product, etc.). The functional unit for measuring the quantity may be separated from the quantity estimation unit 113 as the product measurement unit. The product measuring unit may acquire the quantity of the product measured by a sensor such as a scale, a camera, or a three-dimensional sensor, or may calculate and measure the quantity of the product from the measurement result of the sensor.
Further, each functional unit provided in the control unit 110 of the shortage prevention device 100 and each database stored in the storage unit 120 may be distributed and provided in a plurality of devices.

The present invention can take various other embodiments, and further, various modifications such as omission and substitution can be made without departing from the gist of the present invention. These embodiments and variations thereof are included in the scope and gist of the invention described in the present specification and the like, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

10 Out of stock prevention system (information processing system)
100 Out of stock prevention device (information processing device)
111 Person analysis unit 112 Product registration unit 113 Quantity estimation unit (product measurement unit)
114 Quantity prediction unit 120 Storage unit 121 Program 130 Number of store occupants Database 140 Number of store occupants by day Database 150 Product database 151 Name (product identification information)
154 dimensions (quantity per product)
155 Bulk (quantity per product)
170 Quantity Forecast Database 174 Base Decrease Rate (Past Sales Results)
175 Real decrease rate (product sales volume per hour)
220 Sales floor camera (sensor)
230 Distance measurement sensor (sensor)
240 3D sensor (sensor)

Claims (13)

A product measurement unit that measures the amount of product groups that are a group of products placed in the sales floor,
An information processing apparatus including a quantity estimation unit that estimates the quantity of the product group from the quantity of the product group and the quantity per product. The quantity of the product group is the weight of the product group.
The information processing apparatus according to claim 1, wherein the product measuring unit estimates the amount of the product group from the measurement result of the weighing scale. The quantity of the product group is the bulk of the product group.
The first aspect of claim 1, wherein the product measuring unit estimates the amount of the product group from at least one of an image taken by a camera, a measurement result of a distance measuring sensor, and a measurement result of a three-dimensional sensor. Information processing device. The quantity of the product group is the area occupied by the product group.
The information processing device according to claim 1, wherein the product measuring unit estimates the amount of the product group from an image taken by a camera. The information processing apparatus according to claim 1, further comprising a quantity prediction unit that predicts the quantity of the product group after a predetermined time from the quantity of the product sold per hour and the quantity of the product group. The quantity prediction unit
The information processing apparatus according to claim 5, wherein if the predicted quantity of the product group after the predetermined time is equal to or less than a predetermined predetermined value for each product, an alarm is issued. The quantity prediction unit
The information processing apparatus according to claim 5, wherein the product sales amount per hour is predicted by using past sales results. The quantity prediction unit
Claim 7 is characterized in that, in addition to the past sales performance, the prediction is made using at least one of the store visitor's attributes, weather, time zone, sales performance of the day, and information on events in the commercial area. The information processing device described. The first aspect of claim 1, further comprising a product registration unit that acquires the identification information of the product and the amount of each product from the image taken by the camera and stores the product in the storage unit in association with the identification information of the product. Information processing device. It further includes a product registration unit that acquires the identification information of the product, the amount per product, and the contact information to which the alarm is issued from the image taken by the camera, associates them, and stores them in the storage unit.
The information processing apparatus according to claim 6, wherein the quantity prediction unit issues an alarm to the contact. A program for operating a computer as an information processing device according to any one of claims 1 to 10. A sensor that acquires information on a group of products that are a group of products placed in the sales floor,
A product measurement unit that acquires the quantity of the product group from the information of the product group, and
An information processing system including a quantity estimation unit that estimates the quantity of the product group from the quantity of the product group and the quantity per product. It is an information processing method of an information processing device.
The information processing device
A step to measure the amount of products that are a group of products placed in the sales floor,
An information processing method comprising executing a step of estimating the quantity of the product group from the quantity of the product group and the quantity per product.

Images