JP2023152483A - Information processing system, information processing method and program - Google Patents

Abstract

To provide an information processing system for associating a POS with a table in any situation without storing the layout of the table in a restaurant, and an information processing method and program.SOLUTION: In a restaurant information provision system, a control part of a restaurant information providing server 100 connected to a plurality of user terminals 200A to 200C and a restaurant terminal 300 of a restaurant through the Internet 50 recognizes dishes existing on a plurality of tables in the restaurant from picked-up images of in-store cameras C installed in the restaurant, detects POS information meeting a corresponding recognition result from the POS information of the plurality of tables in the restaurant, and associates the positions of the tables with the dishes recognized in the restaurant with the detected POS information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system, an information processing method, and a program that can perform image recognition processing of food served on a table in a restaurant.

2. Description of the Related Art Conventionally, there has been a technology for recognizing food and drinks that customers are eating and drinking from images captured by cameras in restaurants. For example, Patent Document 1 listed below uses a video acquisition unit that acquires images of eating and drinking spaces, and a learning model that performs machine learning on menu-related progress status (hereinafter referred to as "meal situation") for a group of images of eating and drinking spaces. An image recognition device is disclosed, which includes an image recognition unit that recognizes a meal status from an image acquired by a video acquisition unit, and a status determination unit that determines the progress status of a customer regarding eating and drinking based on the image-recognized meal status. ing.

Re-table 2020/145085 publication

By the way, the recognition of food and drink as in the above-mentioned conventional technology is based on the premise that the layout of each table in a restaurant is associated with the POS (Point of Sale) information of the table. However, it is a heavy burden on restaurants to memorize the layout of each table in the restaurant each time, which can vary depending on the day.

In view of the above circumstances, an object of the present invention is to provide an information processing system, an information processing method, and a program that can link POS and tables without memorizing the layout of tables in a restaurant. be.

In order to solve the above-mentioned problem, an information processing system according to one embodiment of the present invention includes a control unit. The control unit recognizes dishes on a plurality of tables in the restaurant from images captured by a camera installed in the restaurant. The control unit also detects POS information that matches the recognition result from the POS information of a plurality of tables in the restaurant. The control unit then associates the position of the table on which the dish was recognized in the restaurant with the detected POS information.

With this configuration, the information processing system can link the POS and the table in any situation without memorizing the layout of the table in the restaurant by comparing the recognition result of the food on the table with the POS information.

When it is recognized from the captured image that the food has been served within a predetermined time from the order time of the food included in the POS information, the control unit may display the position of the table where the food was served and the POS information. may be linked.

As a result, the information processing system can improve the accuracy of linking by comparing the recognition result of the dish with the POS information while also considering the order time and serving time of the dish.

When the genre of the served dish is recognized, the control unit may link the position of the table and the POS information by comparing the genre with the genre of the dish included in the POS information. good.

As a result, the information processing system recognizes the genre of food and matches it with the genre of food in POS information, eliminating the need for high performance cameras to recognize even the name of the food, and processing that requires heavy processing related to high-resolution images. Since there is no need to do this, the load can be reduced.

The control unit may associate the POS information with the position of the table where the food is served when the same genre as the plurality of food genres included in the POS information is recognized multiple times from the captured image. good.

This allows the information processing system to link the table position and POS information only after multiple dishes of the same genre are collated, making it possible to further improve the accuracy of linking while eliminating the need for high-resolution captured images. can.

When a dish indicated by a dish name included in the POS information is recognized from the captured image, the control unit may associate the POS information with the position of the table where the dish is served.

Thereby, the information processing system can increase the accuracy of linking by recognizing the captured image until the name of the dish is determined.

When a combination of dishes indicated by a plurality of dish names having the same order time included in the POS information is recognized from the captured image, the control unit is configured to determine the location of the table on which the dishes are served and the POS information. You can also tie it.

As a result, the information processing system can further improve the accuracy of linking by linking the table position and POS information only after the captured image and POS information can be compared for multiple dishes ordered at the same time. can.

The control unit recognizes a first dish ordered at a first order time included in the POS information from a first captured image captured at a first time among the captured images, and A second dish ordered at a second order time different from the first order time included in the POS information is captured at a second time different from the first time in the captured image. When the first and second dishes are recognized from the second captured image, the positions of the tables on which the first and second dishes are served may be associated with the POS information.

As a result, the information processing system can further improve the accuracy of linking by linking the table position and POS information only after the captured images and POS information can be compared for multiple dishes ordered at different times. can.

The control unit may exclude the table associated with the POS information from the recognition process from the captured image until the end of accounting for the table.

As a result, in the information processing system, once the linking with POS information is completed, the table is excluded from the recognition process from the captured image until the end of accounting for that table, resulting in a load due to unnecessary recognition process. can be prevented.

The control unit recognizes the user who has arrived at the table and the food eaten by the user from the captured image based on registered data regarding the faces of a plurality of users stored in advance, and the controller recognizes the user who has arrived at the table and the food eaten by the user, and shares the food with the recognized user. Based on this, the user may be associated with the POS information.

As a result, the information processing system can not only simply link tables and POS information, but also grasp which users used the tables and ate which dishes, and use this information for subsequent promotional activities.

In this case, the control unit may further link the position of the recognized user at the table where the user has arrived to the POS information.

Thereby, the information processing system can record which users were seated at which positions at the table as historical information.

An information processing system according to another embodiment of the present invention includes a control section. The control unit recognizes that food has been served on one of the plurality of tables in the restaurant from an image captured by a camera installed in the restaurant. Then, when it is determined that the serving is within a predetermined time from the order time of the dish included in the POS information of any of the tables, the control unit displays the position of the table where the dish was served and the POS information. Link it with.

With this configuration, the information processing system can treat a table on which some kind of food has been served within a predetermined time from the order time at the POS as the table of the orderer, thereby eliminating the need to memorize the layout of the tables in the restaurant and using captured images. It is possible to link the POS and table in any situation without having to recognize the food itself.

The control unit may detect that the food ordered at the first order time included in the POS information has been served within the predetermined time from the first order time at the first time in the captured image. A dish that is recognized from the first captured image taken at , and is included in the POS information, and that is ordered at a second order time that is different from the first order time, is changed from the second order time to the above. If it is recognized from a second captured image taken at a second time different from the first time among the captured images that the food has been served within a predetermined time, each dish is served within a predetermined time. The position of the table and the above POS information may be linked.

As a result, the information processing system links the table position and POS information only after recognizing that multiple dishes ordered at different times have been served within a predetermined time from the order time. Accuracy can be further improved.

An information processing method according to another embodiment of the present invention includes:
Recognizes the food on multiple tables in the restaurant from images captured by a camera installed in the restaurant,
Detecting POS information that matches the recognition result from the POS information of multiple tables in the restaurant,
The method includes associating the position of the table on which the food was recognized in the restaurant with the detected POS information.

An information processing method according to another embodiment of the present invention includes:
Recognizes that the food has been served on one of the multiple tables in the restaurant based on images captured by a camera installed in the restaurant,
If it is determined that the meal was served within a predetermined time from the order time of the food included in the POS information of any of the tables, linking the position of the table where the food was served with the POS information; Including.

A program according to another embodiment of the present invention is provided in an information processing device,
a step of recognizing dishes on a plurality of tables in the restaurant from images captured by a camera installed in the restaurant;
Detecting POS information that matches the recognition result from the POS information of a plurality of tables in the restaurant;
A step of associating the detected POS information with the position of the table in the restaurant where the food was recognized is executed.

A program according to another embodiment of the present invention is provided in an information processing device,
a step of recognizing that food has been served on one of a plurality of tables in the restaurant from an image captured by a camera installed in the restaurant;
If it is determined that the meal was served within a predetermined time from the order time of the food included in the POS information of any of the tables, linking the position of the table where the food was served with the POS information. and execute it.

As explained above, according to the present invention, a POS and a table can be linked in any situation without memorizing the layout of tables in a restaurant. However, this effect does not limit the present invention.

1 is a diagram showing the configuration of a restaurant information providing system according to an embodiment of the present invention. 1 is a diagram showing a hardware configuration of a restaurant information providing server according to an embodiment of the present invention. 1 is a diagram showing the configuration of a database included in a restaurant information providing server according to an embodiment of the present invention. It is a flowchart showing the flow of table/POS linking processing by the restaurant information providing server according to an embodiment of the present invention.

Embodiments of the present invention will be described below with reference to the drawings.

[System configuration]
FIG. 1 is a diagram showing the configuration of a restaurant information providing system according to this embodiment.

As shown in the figure, this system includes a restaurant information providing server 100 on the Internet 50, a plurality of user terminals 200, and a restaurant terminal 300.

The restaurant information providing server 100 is a web server that operates a portal site (restaurant information providing site) that posts information about restaurants. The restaurant information providing server 100 is connected to a plurality of user terminals 200 and a restaurant terminal 300 of a restaurant via the Internet 50.

The restaurant information providing server 100 provides a restaurant information providing system for users of the user terminals 200 on the portal site. Specifically, the restaurant information providing server 100 searches for restaurant information that matches the search conditions based on a search request from the user terminal 200, generates a web page containing the search results, and sends the generated web page to the user terminal 200. Send. The restaurant information providing server 100 also processes reservations for any restaurant from the user terminal 200 of the user who viewed the restaurant information.

The user terminals 200 (200A, 200B, 200C...) are terminals used by users, such as smartphones, mobile phones, tablet PCs (Personal Computers), notebook PCs, desktop PCs, and the like. The user terminal 200 accesses the restaurant information providing server 100, receives the web page, and displays it on the screen using a browser or the like.

The user terminal 200 determines restaurant search conditions based on the user's operations, and transmits a restaurant search request based on the search conditions to the restaurant information providing server 100. Based on the user's operation, the user terminal 200 can transmit a reservation request for one of the restaurants displayed as the search results to the restaurant information providing server 100.

On the other hand, the restaurant information providing server 100 provides a restaurant information management screen (web page) for the restaurants (affiliated stores) listed on the portal site. The user of the restaurant terminal 300 edits and updates the restaurant information (content) on the web page provided to general users as the search result through the management screen, and posts the web page on the portal site. Can be uploaded.

The restaurant terminal 300 is a terminal installed in each restaurant, and is a tablet PC, notebook PC, desktop PC, or the like. The restaurant terminal 300 can perform processing related to its own restaurant information, such as editing and updating the restaurant information, by communicating with the restaurant information providing server 100, based on the operation of the administrator. Although only one restaurant and one restaurant terminal 300 are shown in the figure, there are a plurality of restaurants, and each restaurant has a restaurant terminal 300.

Further, as shown in the figure, an in-store camera C is installed inside the restaurant. One or more in-store cameras C are installed in the restaurant, and are capable of capturing a wide-angle image of the floor on which the tables T in the restaurant are arranged. The in-store camera C is installed on the ceiling or upper part of the wall inside the restaurant, and takes images of a plurality of tables T (foods on top) all at once so as to give a bird's-eye view of the inside of the restaurant. For example, a number or other characters or symbols for identification may be added to the top end of each table T, for example, by a panel or a stamp.

The captured image of the in-store camera C is transmitted to the restaurant terminal 300 via, for example, a wireless relay terminal W installed in the store, and is transmitted from the restaurant terminal 300 to the restaurant information providing server 100. Alternatively, the captured image may be transmitted to the restaurant information providing server 100 without going through the restaurant terminal 300.

When the restaurant information providing server 100 receives an image captured by the in-store camera C, the restaurant information providing server 100 recognizes the dishes on the table T by comparing it with registered data (characteristic data) regarding a plurality of dishes stored in advance, and POS information that matches the recognition result is detected from the POS information in the plurality of tables T. Then, the restaurant information providing server 100 stores the position of the table T on which the above-mentioned dish was recognized in the store in association with the detected POS information. Details of the table/POS linking process will be described later.

[Hardware configuration of restaurant information providing server]
FIG. 2 is a diagram showing the hardware configuration of the restaurant information providing server 100. As shown in the figure, the restaurant information providing server 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an input/output interface 15, and connections between these. A bus 14 is provided.

The CPU 11 accesses the RAM 13 and the like as necessary, and performs various arithmetic operations while collectively controlling each block of the restaurant information providing server 100. The ROM 12 is a nonvolatile memory in which firmware such as an OS, programs, and various parameters to be executed by the CPU 11 are permanently stored. The RAM 13 is used as a work area for the CPU 11, and temporarily stores the OS, various applications being executed, and various data being processed.

A display section 16, an operation reception section 17, a storage section 18, a communication section 19, etc. are connected to the input/output interface 15.

The display unit 16 is a display device using, for example, an LCD (Liquid Crystal Display), an OELD (Organic ElectroLuminescence Display), a CRT (Cathode Ray Tube), or the like.

The operation reception unit 17 is, for example, a pointing device such as a mouse, a keyboard, a touch panel, or other input device. When the operation receiving section 17 is a touch panel, the touch panel can be integrated with the display section 16.

The storage unit 18 is, for example, a nonvolatile memory such as an HDD (Hard Disk Drive), a flash memory (SSD; Solid State Drive), or other solid state memory. The storage unit 18 stores the OS, various applications, and various data.

As will be described later, particularly in this embodiment, the storage unit 18 includes programs such as applications necessary for table/POS linking processing, which will be described later, as well as a restaurant information database, a user information database, and a table history information database. ing.

The communication unit 19 is, for example, various modules for wireless communication such as an Ethernet NIC (Network Interface Card) or a wireless LAN, and is responsible for communication processing between the user terminal 200 and the restaurant terminal 300.

Although not shown, the basic hardware configurations of the user terminal 200 and the restaurant terminal 300 are also substantially the same as the hardware configuration of the restaurant information providing server 100 described above.

[Database configuration of restaurant information providing server]
FIG. 3 is a diagram showing the configuration of a database included in the restaurant information providing server 100.

As shown in the figure, the restaurant information providing server 100 has a restaurant information database 31, a user information database 32, and a visit history information database 33 in the storage unit 18.

The restaurant information database 31 contains, for each restaurant, the restaurant's name, location (address or latitude and longitude) information, area information, access information (nearest station information, walking distance information from the nearest station), telephone number, and ID that identifies the restaurant (store ID), category information of the restaurant's business type and services, information introducing the restaurant (information indicating the characteristics of the store such as store PR text, information on events held by the restaurant, etc.) ), image data related to the restaurant (introducing the restaurant), menu information related to the menu provided by the restaurant, average budget information, business hours, website URL, and other information are stored. This information is input from the restaurant terminal 300 of each restaurant via the management screen provided by the restaurant information providing server 100. The restaurant information database 31 also stores reservation ledger information for each restaurant.

The above menu information is information corresponding to the menus posted on the sites of each restaurant on the above portal site, and stores the menu names of multiple menus that each restaurant can provide for each restaurant. . The menu information may be stored for each menu category, such as appetizer/main, lunch/dinner/course, for example. Furthermore, as menu information, character information indicating menu names, prices, explanations, etc. is stored. In addition, data (dish registration data) for recognizing dishes from images captured by in-store camera C, such as image information such as photos taken of the menu (dishes on the plate) from multiple angles, and characteristic data based thereon. They are stored in association with each other.

The area information includes information regarding a plurality of areas having different size units. Examples of wide areas include prefectures, cities, wards, towns and villages; narrow areas include areas within a few hundred meters from a station (for example, the Ginza area); and areas between these areas include areas within 1 km from a station, etc. Examples include, but are not limited to, an area that is a collection of areas surrounding multiple stations (for example, the "Ginza/Shinbashi/Yurakucho area"). As a result, even the same restaurant can be linked to multiple areas depending on its size.

The category information may include main categories such as Japanese food, Chinese food, Italian food, French food, Yakiniku food, etc., as well as more detailed subcategories such as yakitori and tempura in Japanese food, and pasta and pizza in Italian food.

The above-mentioned reservation ledger information is the reservation information received from the user terminal 200 (or by telephone) via the above-mentioned restaurant information providing site, and includes, for example, the reservation date and time, the reservation user name, the number of reservations, and the reservation information for each restaurant. It has information on tables, reserved courses, etc. As the reservation ledger information, not only reservation information of users who will visit from now on, but also past reservation record information for at least several months is stored. The reservation ledger information can also be accessed from the restaurant terminal 300.

The user information database 32 stores information regarding users who own the user terminals 200 and are users (members) of the restaurant information service via the portal site provided by the restaurant information providing server 100. Specifically, the user information database 32 collects information such as user ID, password, name, email address (other message destination information), phone number, address, age (group), gender, birthday, etc. I remember each time.

Furthermore, the user information database 32 includes face image data of each user and feature data for face authentication obtained by analyzing the face image data (provided by the user). User registration data) may also be stored. In addition to this, face image data of the user wearing a mask and data regarding the user's hairstyle may be stored.

The table history information database 33 stores history information regarding the in-store position information and POS information of the table T linked based on the captured image of the in-store camera C. Specifically, the table history information database 33 stores, for each restaurant, the date and time of visit (time zone), table ID, table position, POS data (for example, information on usage amount, order menu, benefit usage status, etc.), and the number of users. , stay time, etc. are stored in association with each other. POS data is stored from the time an order is started at any table T in each restaurant (including data before checkout).

These databases are mutually referenced and used as necessary in table/POS linking processing by the restaurant information providing server 100, which will be described later.

[Operation of restaurant information providing server]
Next, the operation of the restaurant information providing server 100 configured as above will be explained. The operation is executed in cooperation with hardware such as the CPU 11 and the communication unit 19 of the restaurant information providing server 100 and software stored in the storage unit 18. In the following description, for convenience, the CPU 11 is assumed to be the main operating body.

FIG. 4 is a flowchart showing the flow of table/POS linking processing by the restaurant information providing server 100.

As shown in the figure, the CPU 11 of the restaurant information providing server 100 first determines whether an order has been placed by a customer at any table T (step 41).

Subsequently, the CPU 11 identifies the ordered menu from the POS data (step 42).

Subsequently, the CPU 11 receives the captured image from the in-store camera C (step 43).

Subsequently, the CPU 11 compares the received captured image with dish registration data included in the restaurant information database 31 and corresponding to the menu specified from the POS data (step 44).

Subsequently, the CPU 11 determines whether or not the arrangement of the matched dish on the table T has been recognized (step 45). That is, the CPU 11 is able to recognize the dish indicated by the dish name of the menu specified at the POS from the captured image at a certain time by checking with the dish registration data, and the CPU 11 is able to recognize the dish indicated by the dish name of the menu specified at the POS, and that the dish is brought by an employee and It is determined whether or not it was possible to recognize that the food was served on that table T.

If it is determined that the above-mentioned dish and its serving could not be recognized (No in step 45), the CPU 11 repeats the verification process.

If it is determined that the food and the serving have been recognized (Yes in step 45), the CPU 11 determines whether or not the serving has occurred before a predetermined period of time has elapsed since the order time of the specified menu at the POS (step 46). The predetermined time is, for example, 5 minutes, 10 minutes, 15 minutes, etc., the maximum time required from ordering food to serving, but is not limited thereto.

Note that since the first order at a restaurant is often a drink, the predetermined time may be set to be shorter (for example, 3 minutes, 5 minutes, etc.) for the first menu item ordered. Alternatively, for drinks, the same glass is often used even for different drinks, and image recognition accuracy may be low. May be excluded.

In addition, the CPU 11 not only determines whether the predetermined time has elapsed, but also determines whether the food was served after the first time elapsed from the order time and before the second time elapsed (for example, the order time After 5 minutes have elapsed and before 20 minutes have elapsed). This is because the ordered food is not served immediately after the order is placed, but is served after a period of time equal to the order time plus at least the cooking time.

If it is determined that the meal has not been served after the predetermined time has elapsed (No in step 46), the CPU 11 returns to step 41 and repeats the above process for a new order.

If it is determined that the meal has not been served before the predetermined time has elapsed (Yes in step 46), the CPU 11 determines whether or not this is the second successful recognition of the dish and its serving on table T (step 47). That is, the CPU 11 recognizes that the first dish ordered at the first order time included in the POS data is recognized from the first captured image captured at the first time among the captured images, and A second dish ordered at a second order time after the first order time included in the data was captured at a second time after the first time in the captured image. It is determined whether or not it has been recognized from the second captured image.

If it is determined that the recognition of the dish and its serving is not the second time (it is the first time) (No in step 47), the CPU 11 returns to step 41 and repeats the above process for a new order.

If it is determined that the recognition of the above-mentioned dish and its serving has been successful for the second time (Yes at step 47), the CPU 11 stores information indicating the position of table T in the recognized image in the restaurant and the above-mentioned order. The POS data corresponding to the menu is linked and stored in the table history information database 33 (step 48). Note that the information indicating the position of the table T in the restaurant may be information (numbers, symbols, etc.) indicating the area where the table T is recognized among multiple predetermined areas in the restaurant, or may be information indicating the area where the table T is recognized from among a plurality of predetermined areas in the restaurant, or may be information indicating the area where the table T is recognized from among a plurality of predetermined areas in the restaurant, or may be information indicating the area where the table T is recognized from among a plurality of predetermined areas in the restaurant, or information indicating the area where the table T is recognized from among a plurality of predetermined areas within the restaurant. It may be a sign indicating the position on a map or photograph, or it may be any other information.

As described above, the accuracy of the linking can be further improved by linking the position of the table T and the POS data only after the captured images and POS information can be compared for a plurality of dishes ordered at different times.

Subsequently, the CPU 11 excludes the table T in which the above-mentioned position is linked to the POS data from recognition targets from subsequent captured images (step 49).

Subsequently, the CPU 11 determines whether or not the transaction in the linked table T has been completed (step 50).

If it is determined that the transaction has not been completed (No in step 50), the CPU 11 repeats the determination until the transaction is completed.

If it is determined that the transaction has been completed (Yes in step 50), the CPU 11 includes the excluded table T again in the recognition target (step 51).

In this way, table T, once linked with POS data, is excluded from recognition processing from captured images until the end of accounting for that table, thereby avoiding the load caused by unnecessary recognition processing. It can be prevented.

The CPU 11 repeats the above process every time a customer makes an order and the order is input into the POS information, for the POS information that has not yet been linked to the table position at that time.

[summary]
As described above, according to the present embodiment, the restaurant information providing server 100 stores the layout of the table T by comparing the recognition results of the dishes served on the table T in the restaurant with the POS information. POS information and the position of table T can be linked in any situation without having to do so.

[Modified example]
The present invention is not limited to the above-described embodiments, and may be modified in various ways without departing from the gist of the present disclosure.

In the embodiment described above, the restaurant information providing server 100 even recognized the name of the dish served. However, the restaurant information providing server 100 identifies the genre of the ordered menu included in the POS information without recognizing the name of the dish, and when the genre of the served dish is recognized, the genre and the POS information include the genre. The position of the table T and the POS information may be linked by comparing the genre of the food served. As a result, by recognizing the genre of food and comparing it with the genre of food in POS information, the high performance of the in-store camera C to recognize even the name of the dish is no longer required, and the heavy processing of high-resolution images is no longer required. Since there is no load, the load can be reduced. The genres are, for example, sashimi, pizza, curry, etc. (the dish names are, for example, tuna sashimi, margherita, keema curry, etc.).

In this case, the restaurant information providing server 100 recognizes that the same genre (e.g., pasta) as the genre (e.g., pasta) of the plurality of dishes (e.g., peperoncino, carbonara) included in the POS information has been recognized multiple times from the captured image. In this case, the position of the table where the food is served may be associated with the POS information. Thereby, by linking the table position and POS information only after the same genre has been collated multiple times, it is possible to further improve the accuracy of linking while eliminating the need for high-resolution captured images.

In the above-described embodiment, when a combination of dishes indicated by multiple dish names having the same order time included in the POS information is recognized from the captured image, the restaurant information providing server 100 displays information about the table on which the dishes are served when the combination of dishes indicated by the names of dishes with the same order time included in the POS information is recognized from the captured image. The location and POS information may be linked. That is, when a plurality of dishes ordered at the same time are served at once, the restaurant information providing server 100 executes the above-mentioned linking only when it can recognize them all at once. Thereby, the accuracy of the linking can be further improved compared to the case where a single dish is recognized and the position of the table T and the POS are linked.

In the above-described embodiment, the restaurant information providing server 100 further determines, from the captured image, the user who has arrived at table T and what the user has eaten, based on pre-stored registered data regarding facial features of a plurality of users. A dish may be recognized, and the user and the OS information may be linked based on the recognized user and the dish. As a result, it is possible not only to simply link the table T and POS information, but also to know which user used the table T and ate which dish, which can be used for subsequent promotional activities.

In this case, the restaurant information providing server 100 further links the position of the recognized user at the table T (for example, back left, front right, etc. when viewed from a predetermined reference position) to the POS information. It's okay. Thereby, the information processing system can record which user was seated at which position on the table T as historical information.

In the embodiment described above, the restaurant information providing server 100 executes the linking process when recognizing that the food has been served within a predetermined time from the order time. It is not necessary to perform a determination process. That is, the restaurant information providing server 100 recognizes the food from the captured image, detects POS information (dish name) that matches the dish recognized from the captured image from the POS information of the plurality of tables T, and confirms that the dish is recognized. What is necessary is to link the position of the table T in the restaurant and the detected POS information.

In the embodiment described above, the restaurant information providing server 100 recognized the dish name of the served dish by comparing the dish registration data with the captured image. However, recognition processing for the dish is not essential. That is, the restaurant information providing server 100 recognizes from the captured image of the camera C installed in the restaurant that food has been served on any of the plurality of tables T in the restaurant, and that the food has been served on any of the above tables. If it is determined that a predetermined time has elapsed since the order time of the food included in the POS information of the table, the position of the table where the food was served is linked to the POS information. As a result, a table on which a certain type of food is served within a predetermined time from the order time at POS is regarded as the table of the orderer, thereby eliminating the need to memorize the layout of the tables in the restaurant and recognizing the food itself from the captured image. You can link POS and tables in any situation without having to do anything.

In this case, the restaurant information providing server 100 uses the captured image to indicate that the food ordered at the first order time included in the POS information was served within the predetermined time from the first order time. Among them, dishes that are recognized from the first captured image captured at the first time and that are included in the POS information and that are ordered at a second order time that is different from the first order time are the dishes that are ordered at the second order time, which is different from the first order time. If it is recognized from the second captured image taken at a second time different from the first time among the captured images that the food was served within the predetermined time from the order time, the respective The position of the table T where the food is served may be associated with the POS information. This improves the accuracy of linking by linking the position of table T and POS information only after it is recognized that multiple dishes ordered at different times have been served within a predetermined time from the order time. can be increased.

In the embodiment described above, only one restaurant information providing server 100 is shown, but the processing executed by the restaurant information providing server 100 may be distributed and executed by a plurality of servers. For example, a separate server may be responsible for only the process of matching the captured image of the in-store camera C with the dish registration data.

Furthermore, in the above-described embodiment, the restaurant information providing server 100 on the cloud executed the process of associating the position of the table T with the POS information using the captured image in each restaurant, but each restaurant terminal 300 or A separately provided restaurant server may execute the linking process.

Among the inventions described in the claims of this application, the invention described as an "information processing method" is one in which each step is automatically performed by at least one device such as a computer through information processing using software. Yes, it is not performed by humans using devices such as computers. In other words, the "information processing method" is an information processing method using computer software, and is not a method in which humans operate a calculation tool called a computer.

11...CPU
18...Storage unit 19...Communication unit 31...Restaurant information database 32...User information database 33...Visit history information database 100...Restaurant information providing server 200...User terminal 300...Restaurant terminal C...In-store camera

Claims (16)

Recognizes the food on multiple tables in the restaurant from images captured by a camera installed in the restaurant,
Detecting POS information that matches the recognition result from POS information of a plurality of tables in the restaurant,
linking the position of the table on which the food was recognized in the restaurant and the detected POS information;
An information processing system including a control unit. The information processing system according to claim 1,
When it is recognized from the captured image that the food has been served within a predetermined time from the order time of the food included in the POS information, the control unit may combine the position of the table where the food was served and the POS information. An information processing system that links together. The information processing system according to claim 2,
When the genre of the served dish is recognized, the control unit links the position of the table and the POS information by comparing the genre with the genre of the dish included in the POS information. Information processing. system. The information processing system according to claim 3,
The control unit associates the POS information with the position of the table where the food is served when the same genre as the plurality of food genres included in the POS information is recognized multiple times from the captured image. system. The information processing system according to claim 1 or 2,
The control unit is configured to link the POS information to a position of a table on which the dish is served when a dish indicated by a dish name included in the POS information is recognized from the captured image. The information processing system according to claim 5,
When a combination of dishes indicated by a plurality of dish names having the same order time included in the POS information is recognized from the captured image, the control unit determines the position of the table where the dishes are served and the POS information. Linking information processing system. The information processing system according to any one of claims 1 to 6,
The control unit recognizes a first dish ordered at a first order time included in the POS information from a first captured image captured at a first time among the captured images, and A second dish ordered at a second order time different from the first order time included in the POS information is captured at a second time different from the first time in the captured image. An information processing system that associates the POS information with the position of the table where the first and second dishes are served when the first and second dishes are recognized from the second captured image. The information processing system according to any one of claims 1 to 7,
The control unit excludes a table associated with the POS information from recognition processing from the captured image until the end of accounting for the table. The information processing system according to any one of claims 1 to 8,
The control unit recognizes the user who has arrived at the table and the food eaten by the user from the captured image based on pre-stored registration data regarding the faces of a plurality of users, and identifies the recognized user and the food. An information processing system that links the user and the POS information based on the information. The information processing system according to claim 9,
The control unit further associates the position of the recognized user at the table at which the user is seated with the POS information. Recognizes that the food has been served on one of the multiple tables in the restaurant based on images captured by a camera installed in the restaurant,
If it is determined that the meal was served within a predetermined time from the order time of the food included in the POS information of any of the tables, linking the position of the table where the food was served with the POS information;
An information processing system including a control unit. The information processing system according to claim 11,
The control unit may display a first time in the captured image indicating that the food ordered at the first order time included in the POS information has been served within the predetermined time from the first order time. A dish that is recognized from the first captured image taken at , and is included in the POS information, and that is ordered at a second order time that is different from the first order time, is changed from the second order time to the Each dish is served when it is recognized from a second captured image taken at a second time different from the first time among the captured images that the food was served within a predetermined time. An information processing system that links the table position and the POS information. Recognizes the food on multiple tables in the restaurant from images captured by a camera installed in the restaurant,
Detecting POS information that matches the recognition result from POS information of a plurality of tables in the restaurant,
An information processing method that associates the position of a table in the restaurant where the food is recognized with the detected POS information. Recognizes that the food has been served on one of the multiple tables in the restaurant based on images captured by a camera installed in the restaurant,
If it is determined that the meal was served within a predetermined time from the order time of the food included in the POS information of any of the tables, the position of the table where the food was served is linked to the POS information. Processing method. In the information processing device,
a step of recognizing dishes on a plurality of tables in the restaurant from images captured by a camera installed in the restaurant;
detecting POS information that matches the recognition result from POS information of a plurality of tables in the restaurant;
A program that executes a step of associating a position of a table in which the food is recognized in the restaurant with the detected POS information. In the information processing device,
a step of recognizing that food has been served on one of a plurality of tables in the restaurant from an image captured by a camera installed in the restaurant;
If it is determined that the meal was served within a predetermined time from the order time of the food included in the POS information of any of the tables, linking the position of the table where the food was served with the POS information. A program that runs and .

Images