JP2023146500A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To provide an information processing device, an information processing method, and an information processing program for proposing additional menu orders to customers at appropriate timing.SOLUTION: An information processing device 10 is configured to be able to communicate via a network NW with an imaging device 20 that captures images of food being served to customers, and a clerk terminal 30 used by a clerk who serves the customers. The information processing device includes: an image analysis unit 11 that extracts a remaining amount of the food based on captured image data of the food served on a table captured by the imaging device; a proposal unit 12 that extracts a proposal menu to be proposed to the customers sitting at the table; and a display control unit 13 that displays the proposal menu on the terminal. The proposal unit 12 proposes different proposal menus depending on the type of food and the remaining amount.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

At a restaurant or izakaya, suggesting to customers that they order additional items from the menu may lead to an increase in orders.

Japanese Patent Application Publication No. 2004-206393 Patent No. 6892548

Patent Document 1 describes that images of tableware arranged on a table are photographed, the photographed tableware images are compared with a determination reference image, and if there is a tableware image similar to the determination reference image, the contents of the tableware are determined. An image data processing device is disclosed that determines that the remaining amount of food or drink in the tableware has reached a predetermined amount, and notifies that the amount remaining in the tableware has reached the predetermined amount. Patent Document 2 discloses that one or more recommended dish candidates are statistically selected based on the elapsed time since the user entered the store, and from among the selected recommended dish candidates, the dishes ordered after the user entered the store and the recommended dishes are selected. An apparatus is disclosed that dynamically determines one or more recommended dishes based on correlation with candidates and presents the determined recommended dishes to a user.

An object of the present invention is to provide an information processing device that can suggest additional menu orders to customers at appropriate timing.

In order to achieve the above object, an information processing device according to one aspect of the present invention includes: an image analysis unit that extracts the remaining amount of food based on image data of the food served on the table; The proposal section includes a suggestion section that extracts a suggested menu to be proposed to a customer who eats the food, and a display control section that displays the suggested menu on a terminal, and the suggestion section selects different menus according to the type of the dish and the remaining amount. Extract the suggestion menu.

The display control unit may display identification information of the table on the terminal together with the suggestion menu.

The display controller further includes a history recording unit that acquires information indicating that the proposal of the proposal menu has been implemented from the terminal and records it in association with the table and the proposal menu, and the display control unit is configured to acquire information that the proposal of the proposal menu has been implemented. The proposed menu without a record of this may be displayed on the terminal.

In order to achieve the above object, an information processing method according to another aspect of the present invention performs an image analysis process on a computer to extract the remaining amount of the dish based on image data of the dish being served on the table. , a proposal process for extracting a proposed menu to be proposed to the customers sitting at the table, and a display control process for displaying the proposed menu on a terminal, and in the proposal process, according to the type of the dish and the remaining amount. , extracting the different proposed menus.

In order to achieve the above object, an information processing program according to yet another aspect of the present invention includes an image analysis process for extracting the remaining amount of the dish based on image data of the dish being served on the table; A computer is caused to execute a proposal process for extracting a proposed menu to be proposed to customers sitting at the table, and a display control process for displaying the proposed menu on a terminal, Then, different proposed menus are extracted.
Note that the computer program can be provided while being stored in various data-readable recording media, or can be provided in a downloadable manner via a network such as the Internet.

According to the present invention, it is possible to suggest additional menu orders to the customer at an appropriate timing.

FIG. 1 is a functional block diagram showing the functions of the information processing apparatus according to the first embodiment of the present invention and each component connected through a network. FIG. 4 is a diagram showing (a) an example of a table stored in a menu database (DB), and (b) an example of a table stored in a storage unit included in the information processing device. FIG. 2 is a sequence diagram showing the flow of processing executed by the information processing device. FIG. 2 is a diagram showing the correspondence between a schematic diagram showing the state of tables in a restaurant and examples of screens displayed on a staff terminal by the information processing device, (a) a first example, (b) a second example, (c) A diagram showing a third example.

Embodiments of an information processing apparatus according to the present invention will be described below with reference to the drawings.

●Information processing equipment●
FIG. 1 shows the configuration of an information processing device 10 according to an embodiment of the present invention. The information processing device 10 is a device that extracts a menu to be proposed to customers eating and drinking at the table, based on the type and remaining amount of food on the photographed table.

As shown in FIG. 1, the information processing device 10 is capable of communicating via the network NW with at least an imaging device 20 that captures images of dishes being served to customers, and a clerk terminal 30 used by a store clerk who serves customers. It is composed of Further, the information processing device 10 is communicably connected to a menu database (DB) 60 that stores at least menus provided at the restaurant and suggested menus for customers who eat and drink the menus.

The information processing device 10 may be configured by a hardware device, or some or all of its functions may be realized by a cloud computer. Further, the information processing device 10 may be configured with a plurality of hardware configurations. In this case, the plurality of hardware configurations may be connected by wire or wirelessly and may exchange information with each other. Furthermore, each configuration of the information processing device 10 may be realized by an API (Application Programming Interface).

Although communication among the information processing device 10, the imaging device 20, the clerk terminal 30, and the menu DB 60 is wireless in this embodiment, some or all of the connections may be wired.
Further, the information processing device 10 may constitute an information processing system together with some terminals among the imaging device 20, the clerk terminal 30, and the menu DB 60.

●Imaging device 20
The imaging device 20 is a device that captures images of food being served to customers eating at a restaurant. The RAM (Random Access Memory) and ROM (Read Only Memory) that store the information constitute a functional block mainly consisting of the imaging section 21 and the communication processing section 22. Note that although one imaging device 20 is depicted in FIG. 1, a plurality of imaging devices 20 may be connected to one information processing device 10.

The imaging unit 21 is a functional unit that captures images of food being served to customers eating at a restaurant, and is, for example, an omnidirectional camera. Further, the omnidirectional camera may be a so-called omnidirectional AI camera equipped with AI. The imaging unit 21 is attached to the ceiling, for example, and images the food on the table from above.

The communication processing unit 22 is a processing unit that is capable of transmitting and receiving data according to a predetermined protocol with the information processing device 10 via a network NW such as the Internet, and transmits captured image data and the imaging device 20. and identification information are associated with each other and transmitted to the information processing device 10. The imaging data is transmitted, for example, approximately regularly.

●Staff terminal 30
The clerk terminal 30 is a terminal owned by a clerk who mainly serves customers. The clerk terminal 30 mainly includes a display section 31, an input section 32, and a communication processing section 33.
The display unit 31 is realized by a display or the like for outputting data. The display unit 31 displays, for example, a list of menus available at the connected restaurant, order history, and menus proposed at the table.

The input unit 32 is realized by a touch panel, keyboard, mouse, camera, microphone, or the like for inputting data. The input unit 32 can input menu orders for each table, for example.

The communication processing unit 33 is a processing unit that is capable of transmitting and receiving data according to a predetermined protocol with the information processing device 10 via a network NW such as the Internet. Receive identification information. Further, the clerk terminal 30 transmits to the information processing device 10, via the communication processing unit 33, information indicating that a proposal has been made to the table.

●Menu DB60
The menu DB 60 is a database in which types of dishes, extraction conditions for proposals based on the amount remaining in the table, and information on menus to be proposed are stored in association with each other.
FIG. 2A is a diagram showing an example of the proposal table T1 stored in the menu DB 60. In the suggestion table T1, menu names or identification information and product images are linked. This product image is referred to by the image analysis unit 11, which will be described later, when identifying the food on the table. Further, the proposal table T1 may store various images of remaining capacity, images of empty menus, etc. for each menu.

In addition, the menu DB 60 stores suggested menus to be proposed to customers who have ordered each menu, and thresholds for remaining amounts at which the proposed menus are proposed. Note that the remaining amount conditions shown in FIG. 2(a) are all satisfied when the remaining amount is below a predetermined threshold, but the remaining amount condition is not limited to this. Alternatively, the proposed menu may be extracted when the remaining amount is within a predetermined range.

The suggested menu may be a menu that pairs well with the ordered menu, or may be a recommended menu after eating or drinking the ordered menu. The proposed menu may be the same menu as the ordered menu, that is, a suggestion for a second helping, or it may be different from the ordered menu. For example, in a condition where the remaining amount threshold is medium or higher, a menu that is assumed to be eaten at the same time as the ordered menu is associated, while in a condition where the remaining amount threshold is below a predetermined value. , menus are associated with the assumption that the ordered menu will be eaten and eaten after being consumed. For example, if the remaining amount of beer is 90% or less, the proposed menu is a snack, but if the remaining amount is 30% or less, a second beer or sake as another drink. is stored as a suggestion menu.

The suggestion menu may be stored in advance and not changed, or may be manually rewritten. The proposed menu may be automatically changed by analyzing the order history at the store or a plurality of predetermined stores.

The menu DB 60 may store different proposal tables T1 for each restaurant. The menu DB 60 may store settings for the presence or absence of notifications for menus for each restaurant. In the example of FIG. 2(a), the setting is such that notification is not performed for hamburgers regardless of the remaining amount. Further, the menu DB 60 may store settings for notification presence/absence for each notification condition based on the remaining amount, for example, for each restaurant.

Furthermore, in the menu DB 60, a plurality of proposed menus may be associated with one menu and remaining amount notification condition. In the example of FIG. 2(a), when the beer content is 90% or less, snacks 1 and 2 are extracted as suggested menus.

●Information processing device 10
Here, the configuration of the information processing device 10 will be explained using FIG. 1. As shown in FIG. 1, the information processing device 10 mainly includes an image analysis section 11, a proposal section 12, a display control section 13, a history recording section 14, an order reception section 15, a communication processing section 16, and a storage section 17. Configure functional blocks.

The image analysis unit 11 is a functional unit that performs image analysis on the image data acquired via the communication processing unit 16 and extracts the remaining amount of each dish being served. The image analysis unit 11 identifies the identification information of the table and the area on the table where the food is placed. The image analysis unit 11 also compares the menu image stored in the menu DB 60 with the image of the food reflected in the imaged data, and identifies the dish reflected in each region of the imaged data. The image analysis unit 11 may also refer to the order history of menus ordered at the table, and collate the menus included in the order history with the imaging data of the table to identify the dishes on the table. The image analysis unit 11 can identify the dish using an appropriate method such as pattern matching or object recognition technology.

The image analysis unit 11 also extracts the remaining amount of each dish reflected in the imaged data. The image analysis unit 11 may extract the remaining amount of each dish by comparing with images of various remaining amounts linked to the proposal table T1, for example. The remaining amount is calculated, for example, as a percentage of the total amount as 100%. The image analysis unit 11 may calculate the remaining amount by setting the amount at the time of serving analyzed from the imaging data as 100%. According to this configuration, even if the amount of food is a little low from the time of serving, it is possible to accurately calculate the percentage of remaining amount.

The proposal unit 12 is a functional unit that refers to the menu DB 60 and extracts a proposed menu to be proposed to customers seated at the table based on the type and remaining amount of food on the table in the imaging data. The suggestion unit 12 extracts different suggested menus depending on the type of dish and the remaining amount. The suggestion unit 12 extracts a suggested menu when the remaining amount of food on the table satisfies a predetermined condition stored in the suggestion table T1.

The suggestion unit 12 may extract a proposed menu when a plurality of the same dishes are served and the remaining amount of at least one dish satisfies a condition.

Further, the proposal unit 12 may refer to the remaining amount of a plurality of dishes to extract a proposed menu that is different from the case where the remaining amount of one dish satisfies the condition. For example, in a case where a plurality of the same dishes are being served, if the remaining amount of all the beers served at the table is below a predetermined value, an additional order for pitchers may be suggested. Further, for example, the remaining amounts of multiple types of dishes may be referred to, and if the remaining amounts of all the dishes are below a predetermined value, an after-meal dessert may be suggested. Furthermore, the suggestion unit 12 may extract the consumption rate from the remaining amount periodically acquired by the image analysis unit 11, and may extract different suggested menus based on the consumption rate. For example, if the beer consumption rate is higher than a predetermined value, as an exception, water may be extracted instead of suggesting a refill of beer. The extracted suggested menu is transmitted to the clerk terminal 30 together with the table identification information.

The display control unit 13 is a functional unit that causes the store clerk terminal 30 to display the proposed menu extracted by the proposal unit 12. The display control unit 13 notifies the clerk terminal 30 that the proposed menu has been extracted. The display control unit 13 may also cause the clerk terminal 30 to display the identification information of the table whose remaining amount meets a predetermined condition.

Note that when the proposal unit 12 extracts a plurality of the same proposed menus from the same table, the display control unit 13 may combine the notifications into a single notification without performing notification according to the number of extractions. Furthermore, it is not necessary to display the number of dishes that meet the conditions. If the same suggested menu is extracted from the same table after being notified of a predetermined suggested menu and before the store clerk makes a suggestion, the display control unit 13 may omit the notification regarding the extraction. According to this configuration, excessive notification to the clerk terminal 30 can be suppressed.

Further, in addition to displaying the proposed menu, the clerk terminal 30 also displays an operator for accepting an order for the proposed menu, and by selecting the operator, it is possible to accept an order for the proposed menu for the proposed table. It may be . According to this configuration, the clerk can make a proposal to the table with the proposal menu displayed on the clerk terminal 30 and input the received order as is, which is highly convenient.

In place of or in addition to the display on the staff terminal 30, the display control unit 13 displays information on a desk terminal linked to the table, a customer terminal held by a customer at the table, or a display posted inside the restaurant. The suggested menu may be displayed on a display or the like.

The history recording unit 14 is a functional unit that records an order history, a remaining amount detection history, and a menu proposal history in the storage unit 17. The history recording unit 14 stores the history in the history table T2 shown in FIG. 2(b), which is stored in the storage unit 17.

As shown in FIG. 2(b), the history table T2 stores a table number (an example of table identification information), order history, and current remaining amount. The order history may store menus received by the order reception unit 15, which will be described later. Moreover, only menus for which suggested menus based on the remaining amount are set from among the received menus may be extracted and stored in the history table T2. The current remaining amount stores the remaining amount analyzed by the image analysis unit 11.

The proposal history stores the history of proposed menus displayed on the clerk terminal 30 upon detection of the remaining amount of the menu. The proposal history is stored, for example, when display control is performed on the clerk terminal 30 by the display control unit 13. In addition, in the suggestion column, it is recorded whether or not the clerk actually made a suggestion to the customer. The proposal presence/absence column is stored, for example, by inputting that the proposal has been made on the clerk terminal 30.

The display control unit 13 refers to the history table T2 and causes the clerk terminal 30 to display a suggestion menu for which there is no record that a suggestion has been made. That is, the display control unit 13 does not display the proposed menu that has already been proposed on the clerk terminal 30. According to this configuration, it is possible to prevent the same proposal from being made to the same customer. This configuration is also useful when a plurality of employee terminals 30 are operating within one restaurant. That is, according to this configuration, it is possible to prevent a store clerk different from the store clerk who made the proposal from making the same proposal to the same customer.

The order receiving unit 15 is a functional unit that receives orders input from the clerk terminal 30.
The order reception unit 15 associates the identification information of the table where the order was placed with the order details and stores them in the history table T2 of the storage unit 17.

The communication processing unit 16 is a processing unit that is capable of transmitting and receiving data according to a predetermined protocol with the imaging device 20 and the store clerk terminal 30 via a network NW such as the Internet. The communication processing unit 16 receives information related to menus from the menu DB 60. The communication processing unit 16 also transmits menu information to the clerk terminal 30 and receives orders from the clerk terminal 30. Furthermore, the communication processing unit 16 receives imaging data from the imaging device 20.

●Processing Flow As shown in FIG. 3, first, the clerk terminal 30 receives an order input from a customer (step S101). Information regarding the accepted order is transmitted to the information processing device 10. This information may be transmitted from the information processing device 10 to an appropriate kitchen terminal installed in the kitchen and communicated to the person in charge of cooking. Further, this information may be sent to an appropriate order processing device instead of the information processing device 10. Information related to the order is stored as an order history in the history table T2 in association with table identification information (step S102).

The imaging device 20 images the food on the table (step S103). The imaging data is transmitted to the information processing device 10 at an appropriate timing (step S104). Next, the information processing device 10 receives this, and the image analysis unit 11 performs image analysis to extract the remaining amount of each dish (step S105). Next, referring to the menu DB 60, if the remaining amount satisfies the conditions set for the dish (Y in step S106), the process moves to step S107. If the remaining amount does not satisfy the predetermined condition (N in step S106), the process returns to step S103.

Next, the information processing device 10 extracts a proposed menu that meets the conditions (step S107), and transmits information on the proposed menu to the clerk terminal 30 together with the table identification information. The clerk terminal 30 displays the proposed menu and table identification information (step S108). When the clerk terminal 30 receives an input from the clerk indicating that the proposed menu has been proposed to the customer (step S109), the clerk terminal 30 transmits the information to the information processing device 10. In response to this input, the display of the suggestion menu on the clerk terminal 30 disappears. The information processing device 10 stores the proposal history in the history table T2 in association with the proposed proposal menu and the order that triggered the proposal (step S110).

●Screen example Here, the situation on the table and the screen example displayed on the clerk terminal 30 will be explained using FIG. 4.
FIG. 4(a) shows table 1001, where four beers are served. Further, two of the beers have a remaining amount of 100%, and the remaining amount of the remaining two beers has a remaining amount of 30%. In this case, as shown on the right side of the figure, a message to the effect that beer should be proposed to table 1001 is displayed on the clerk terminal 30. Furthermore, although there is only one beer that meets the notification conditions, the clerk terminal 30 is notified only once.

FIG. 4(b) shows table 1002, where two wines are served. The remaining amount of one wine is 100%, and the remaining amount of the remaining one is 10%. In this case, as shown on the right side of the figure, a message to the effect that wine should be proposed to table 1002 is displayed on the clerk terminal 30.

FIG. 4(c) shows the state of table 1003, where fried chicken and hamburgers are served, both of which have 0% remaining. Here, as shown in the proposal table T1 in FIG. 2(a), ice cream is extracted according to the remaining amount of fried chicken, and a message is displayed on the clerk terminal 30 to the effect that ice cream should be proposed to table 1003. be done. Also, since notifications for hamburgers are not set, there is no indication of how much hamburger remains.

As described above, according to the information processing device according to the present invention, it is possible to suggest additional menu orders to the customer at an appropriate timing.

10 Information processing device 11 Image analysis section 12 Proposal section 13 Display control section 14 History recording section 15 Order reception section 16 Communication processing section 17 Storage section 20 Imaging device 30 Staff terminal 60 Menu database (menu DB)

Claims (5)

an image analysis unit that extracts the remaining amount of the food based on image data of the food served on the table;
a proposal unit that extracts a proposed menu to be proposed to customers sitting at the table;
a display control unit that displays the proposed menu on a terminal;
Equipped with
The suggestion unit extracts different suggested menus depending on the type of dish and the remaining amount;
Information processing device.
The display control unit causes identification information of the table to be displayed on the terminal together with the suggestion menu.
The information processing device according to claim 1.
further comprising a history recording unit that acquires information that the proposal of the proposal menu has been implemented from the terminal and records it in association with the table and the proposal menu,
The display control unit causes the terminal to display the proposal menu for which there is no record that the proposal has been implemented.
The information processing device according to claim 1 or 2.
to the computer,
Image analysis processing for extracting the remaining amount of the food based on image data of the food served on the table;
a proposal process for extracting a proposed menu to be proposed to customers sitting at the table;
Display control processing for displaying the proposed menu on a terminal;
including;
In the proposal processing, different proposed menus are extracted depending on the type of dish and the remaining amount;
Information processing method.
Image analysis processing for extracting the remaining amount of the food based on image data of the food served on the table;
a proposal process for extracting a proposed menu to be proposed to customers sitting at the table;
Display control processing for displaying the proposed menu on a terminal;
make the computer run
In the proposal processing, different proposed menus are extracted depending on the type of dish and the remaining amount;
Information processing program.

Images