JP7168110B1 - Information processing system, information processing method and program - Google Patents

Abstract

A customer visiting a restaurant is associated with a table used by the customer. An information processing system has a control unit. The control unit captures a first image captured by a first camera that is installed at the entrance of the restaurant and is capable of capturing the faces of users visiting the restaurant from almost the front, and registered data about the faces of a plurality of users stored in advance. to identify the user who has visited the store. Further, the control unit is based on a second captured image captured by a second camera installed in the restaurant and capable of capturing an image of the floor on which the tables in the restaurant are arranged, and the registration data of the specified user. Next, identify the table where the identified user has arrived. Then, the control unit associates and stores the user identification information for identifying the specified user and the table identification information for identifying the specified table. [Selection drawing] Fig. 4

Description

The present invention relates to an information processing system, an information processing method, and a program capable of executing authentication processing of a user visiting a restaurant.

Conventionally, there has been known a technique of recognizing congestion in a store using a sensor such as a camera. For example, in Patent Document 1 below, marks are placed in advance on the floor in the store, the floor in the store is photographed with a CCD camera, the number of marks recognized in the image portion of the floor in the photographed image is counted, It describes calculating the degree of congestion in the store using the counted number of marks.

Japanese Patent No. 5354697

However, in the conventional technology described above, the cameras installed in the store are often installed so as to capture a bird's-eye view of the store. Accuracy is often insufficient to identify the users of each table.

In view of the circumstances as described above, an object of the present invention is to provide an information processing system, an information processing method, and a program capable of associating a user visiting a restaurant with a table used by the user. .

In order to solve the above problems, an information processing system according to one aspect of the present invention has a control unit. The control unit receives a first captured image captured by a first camera that is installed at the entrance of the restaurant and is capable of capturing the faces of users visiting the restaurant from almost the front, and registered data regarding the faces of a plurality of users stored in advance. to identify the user who has visited the store. Further, the control unit is based on a second captured image captured by a second camera installed in the restaurant and capable of capturing an image of the floor on which the tables in the restaurant are arranged, and the registration data of the specified user. Next, identify the table where the identified user has arrived. Then, the control unit associates and stores the user identification information for identifying the specified user and the table identification information for identifying the specified table.

As a result, the information processing system identifies the visitor from the captured image of the camera at the entrance with high authentication accuracy, and transfers the specified information to the camera inside the store, so that even from the captured image of the camera inside the store with relatively low authentication accuracy The user can be detected and the user and the table used by the user can be linked. The information processing system is composed of one or more information processing devices.

The control unit may associate and store the user identification information of the specified user and the POS data of the specified table.

As a result, the information processing system can collect user attribute information such as which user visited the store, what menu they ordered, and how much they spent.

The control unit selects the identified user from the user identification target by the first captured image or the user identification target by the second captured image from the time when the specified user is specified until the user leaves the store. may be excluded.

As a result, the information processing system can reduce the load by omitting useless processing by excluding the identified user from identification processing by the camera at the entrance until the user leaves the store.

The control unit determines that a plurality of users are identified based on the first captured image, and that the plurality of users arrive based on the second captured image and the registration data of the identified plurality of users. When specifying each of the tables, if the table at which one of the plurality of users has arrived can be specified, the table is specified by the second captured image until the user leaves the store. The user may be excluded from the target of processing.

As a result, even when table identification processing by the second captured image for the user identified by the first captured image is accumulated for a plurality of users, the information processing system can select one of the plurality of users from the second captured image. is detected, the registration data of that user is excluded from the verification target of the subsequent specific processing, so that unnecessary processing can be eliminated and the load can be reduced.

Specifically, the control unit determines whether the specified user is detected based on a third captured image captured by a third camera installed at the exit of the restaurant and capable of capturing the face of the user leaving the restaurant from substantially the front. It may be determined whether or not the customer has left the store.

Alternatively, the control unit may determine whether the specified user has left the store based on POS data in a table associated with the specified user.

The imaging range of the first camera and the imaging range of the second camera may not overlap.

As a result, even if the imaging ranges of the first camera and the second camera do not overlap, the information processing system can link the authentication information based on the first captured image and the second captured image to create a table used by the user. can be specified.

The first camera may capture an image of the user's face from a short distance, and the second camera may capture an image of the inside of the restaurant from a bird's-eye view.

As a result, even if the first camera and the second camera have the same performance, the user's face recognition accuracy differs. Therefore, the image captured by the first camera has a function of assisting face authentication processing based on the image captured by the second camera.

The control unit compares the second captured image with the registered data of the specified user, and compares the image other than the face of the user included in the first captured image with the second captured image. The table at which the identified user has arrived may be identified by matching with the included image other than the user's face.

As a result, even if the user's face cannot be detected from the image captured by the camera in the store, the information processing system can detect the user from parts other than the user's face (hairstyle, etc.), the user's clothes, and belongings (bag, etc.). becomes possible.

The control unit detects the specified user from a second captured image of a second camera capable of capturing an area between the table and the entrance, among the plurality of second cameras, and detects the specified user. You may specify the table you arrived at.

As a result, the information processing system can reduce the load by omitting unnecessary processing by limiting the use of cameras among the plurality of in-store cameras that are highly likely to detect the specified user.

An information processing method according to another aspect of the present invention comprises:
Based on the first captured image captured by a first camera installed at the entrance of the restaurant and capable of capturing the faces of the users visiting the restaurant almost from the front, and the pre-stored registration data regarding the faces of a plurality of users, the above Identify users who have visited the store,
Based on the second captured image captured by a second camera installed in the restaurant and capable of capturing the floor on which the tables in the restaurant are arranged, and the registered data of the identified user identify the table the user arrived at,
and storing the user identification information for identifying the specified user and the table identification information for identifying the specified table in association with each other.

A program according to another aspect of the present invention is provided in an information processing device,
Based on the first captured image captured by a first camera installed at the entrance of the restaurant and capable of capturing the faces of the users visiting the restaurant almost from the front, and the pre-stored registration data regarding the faces of a plurality of users, the above identifying a user who has visited the store;
Based on the second captured image captured by a second camera installed in the restaurant and capable of capturing the floor on which the tables in the restaurant are arranged, and the registered data of the identified user identifying the table at which the user has landed;
a step of correlating and storing user identification information for identifying the identified user and table identification information for identifying the identified table.

As described above, according to the present invention, it is possible to associate a customer visiting a restaurant with a table used by the customer. However, this effect does not limit the invention.

It is a figure showing composition of a restaurant information offer system concerning one embodiment of the present invention. It is a figure showing hardware constitutions of a restaurant information offer server concerning one embodiment of the present invention. 4 is a diagram showing the structure of a database of a restaurant information providing server according to one embodiment of the present invention; FIG. 10 is a flow chart showing the flow of user-table linking processing by the restaurant information providing server according to one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[System configuration]
FIG. 1 is a diagram showing the configuration of a restaurant information providing system according to this embodiment.

As shown in the figure, this system includes a restaurant information providing server 100 on the Internet 50 , a plurality of user terminals 200 and a restaurant terminal 300 .

The restaurant information providing server 100 is a web server that operates a portal site (restaurant information providing site) on which information on restaurants is posted. The restaurant information providing server 100 is connected to a plurality of user terminals 200 , restaurant terminals 300 of restaurants, and digital signage terminals 400 via the Internet 50 .

The restaurant information providing server 100 provides a restaurant information providing system for users of the user terminals 200 on the portal site. Specifically, the restaurant information providing server 100 searches for restaurant information that matches the search conditions based on a search request from the user terminal 200, generates a web page on which the search results are posted, and sends it to the user terminal 200. Send. Also, the restaurant information providing server 100 acts as a proxy for accepting reservations for any restaurant from the user terminal 200 of the user who browses the restaurant information.

User terminals 200 (200A, 200B, 200C, . . . ) are terminals used by users, such as smartphones, mobile phones, tablet PCs (Personal Computers), notebook PCs, desktop PCs, and the like. The user terminal 200 accesses the restaurant information providing server 100, receives the web page, and displays it on the screen using a browser or the like.

The user terminal 200 determines search conditions for restaurants based on the user's operation, and transmits a restaurant search request based on the search conditions to the restaurant information providing server 100 . Then, the user terminal 200 can transmit a reservation request for one of the restaurants displayed as the search results to the restaurant information providing server 100 based on the user's operation.

On the other hand, the restaurant information providing server 100 provides restaurant information management screens (web pages) for restaurants (affiliated restaurants) posted on the portal site. The user of the restaurant terminal 300 edits and updates the restaurant information (contents) on the web page provided to general users as the search result via the management screen, and uploads the web page to the portal site. can be uploaded.

The restaurant terminal 300 is a terminal installed in each restaurant, and is a tablet PC, a notebook PC, a desktop PC, or the like. The restaurant terminal 300 can execute processing related to its own restaurant information, such as editing and updating of the restaurant information, through communication with the restaurant information providing server 100 based on the operation of the administrator. Although only one restaurant and one restaurant terminal 300 are shown in the figure, there are a plurality of restaurants, and each restaurant has a restaurant terminal 300 .

As shown in the figure, an entrance camera (first camera) Cd is installed at the entrance of the restaurant, and an in-store camera (second camera) Ci is installed inside the restaurant. The entrance camera Cd is installed at the entrance of the restaurant, and can image the face of the user visiting the restaurant almost from the front. One or a plurality of in-store cameras Ci are installed in the restaurant, and can take a wide-angle image of the floor on which the tables T are arranged in the restaurant. The in-store camera Ci is installed on the ceiling or the upper part of the wall inside the restaurant, and collectively images a plurality of tables T so as to overlook the inside of the restaurant. In this embodiment, it is assumed that the image captured by the in-store camera Ci includes the range captured by the entrance camera Cd. However, in the present invention, the captured images of the entrance camera Cd and the in-store camera Ci do not have to overlap each other. Each table T may have, for example, an identification number or other characters or symbols attached to it, for example, by means of a panel or engraving. The entrance camera Cd captures the user's face from a short distance, and the in-store camera Ci captures the user's face from a long distance. Therefore, even if the cameras have the same performance, the accuracy of the user's face authentication differs. .

Images captured by the entrance camera Cd and the in-store camera Ci are transmitted to the restaurant terminal 300 via, for example, a wireless relay terminal W installed in the store, and then transmitted from the restaurant terminal 300 to the restaurant information providing server 100 . Alternatively, the captured image may be transmitted to the restaurant information providing server 100 without going through the restaurant terminal 300 .

Although only one restaurant is shown in the figure, there may be a plurality of restaurants, each of which is provided with the restaurant terminal 300, the entrance camera Cd, and the in-store camera Ci.

When the restaurant information providing server 100 receives the captured image (first captured image) of the entrance camera Cd, it compares it with registered data (feature data) relating to the faces of a plurality of users stored in advance, thereby identifying the restaurant. Identify users who have visited the store. Further, when the restaurant information providing server 100 receives the captured image (second captured image) of the in-store camera Ci, it compares the captured image with the registered data of the specified user to obtain the table where the user has arrived. Identify T. The restaurant information providing server 100 stores the user identification information (user ID) identifying the specified user and the table identification information (table ID) identifying the specified table T in association with each other. Furthermore, the restaurant information providing server 100 stores the user identification information of the specified user and the POS data of the specified table T in association with each other. The details of the user-table linking process will be described later.

[Hardware configuration of restaurant information providing server]
FIG. 2 is a diagram showing the hardware configuration of the restaurant information providing server 100. As shown in FIG. As shown in the figure, the restaurant information providing server 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an input/output interface 15, and these are connected to each other. A bus 14 is provided.

The CPU 11 appropriately accesses the RAM 13 or the like as necessary, and performs overall control of each block of the restaurant information providing server 100 while performing various kinds of arithmetic processing. The ROM 12 is a non-volatile memory that permanently stores firmware such as an OS, programs, and various parameters to be executed by the CPU 11 . The RAM 13 is used as a work area for the CPU 11, and temporarily holds the OS, various applications being executed, and various data being processed.

The input/output interface 15 is connected to a display unit 16, an operation reception unit 17, a storage unit 18, a communication unit 19, and the like.

The display unit 16 is a display device using, for example, an LCD (Liquid Crystal Display), an OELD (Organic ElectroLuminescence Display), a CRT (Cathode Ray Tube), or the like.

The operation reception unit 17 is, for example, a pointing device such as a mouse, a keyboard, a touch panel, or other input devices. When the operation reception unit 17 is a touch panel, the touch panel can be integrated with the display unit 16 .

The storage unit 18 is, for example, a non-volatile memory such as an HDD (Hard Disk Drive), a flash memory (SSD; Solid State Drive), or other solid state memory. The storage unit 18 stores the OS, various applications, and various data.

As will be described later, particularly in this embodiment, the storage unit 18 stores programs such as applications necessary for user-table linking processing based on the congestion situation and reservation status of restaurants, which will be described later, as well as a restaurant information database and user information. It has a database and a visit history information database.

The communication unit 19 is, for example, a NIC (Network Interface Card) for Ethernet, various modules for wireless communication such as a wireless LAN, and performs communication processing between the user terminal 200, the restaurant terminal 300, and the digital signage terminal 400. bear.

Although not shown, the basic hardware configurations of the user terminal 200 and the restaurant terminal 300 are substantially the same as the hardware configuration of the restaurant information providing server 100 .

[Database configuration of restaurant information providing server]
FIG. 3 is a diagram showing the structure of the database of the restaurant information providing server 100. As shown in FIG.

As shown in the figure, the restaurant information providing server 100 has a restaurant information database 31 , a user information database 32 and a visit history information database 33 in the storage unit 18 .

The restaurant information database 31 stores, for each restaurant, the name of the restaurant, location (address or latitude and longitude) information, area information, access information (nearest station information, walking distance information from the nearest station), telephone number, ID that identifies the restaurant (store ID), category information of the restaurant's business type and service, information introducing the restaurant (information indicating the characteristics of the restaurant such as PR text of the restaurant, event information held by the restaurant, etc.) ), image data relating to restaurants (introducing restaurants), menu information relating to menus provided by restaurants, average budget information, business hours, website URLs, and other information. These pieces of information are input from the restaurant terminal 300 of each restaurant through the management screen provided by the restaurant information providing server 100 . The restaurant information database 31 also stores reservation ledger information for each restaurant.

The menu information is information corresponding to the menu posted on the site of each restaurant on the portal site, and stores the menu names of a plurality of menus that each restaurant can provide for each restaurant. . The menu information may be stored for each menu category such as appetizer/main dish and lunch/dinner/course. As the menu information, in addition to character information indicating menu names, prices, explanations, etc., image information such as photographs taken of the menu is also stored in association with the menu information.

The area information includes information on a plurality of areas with different size units. A wide area is, for example, a prefecture or municipality, a narrow area is, for example, within several hundred meters from a station (for example, the "Ginza area"), and an area between them is, for example, an area within 1 km from a station. , an area where a plurality of station surrounding areas are grouped together (for example, “Ginza/Shimbashi/Yurakucho area”), etc., but not limited to these. As a result, even the same restaurant is linked to multiple areas depending on its size.

Further, information (terminal ID, etc.) related to the digital signage terminal 400 installed in the area is associated with the area (for example, the narrowest area among a plurality of areas), and the digital signage is connected via the area. The terminal 400 and each restaurant are associated with each other. However, apart from the area information, the terminal ID of the digital signage terminal 400 and the store ID of the restaurant may be directly associated.

The category information may include, for example, main categories such as Japanese food, Chinese food, Italian food, French food, and yakiniku meat, and more detailed sub-categories such as yakitori/tempura in Japanese food and pasta/pizza in Italian food.

The reservation ledger information is reservation information received from the user terminal 200 (or by telephone) via the restaurant information providing site. It has information such as tables and reserved courses. As the reservation ledger information, not only the reservation information of the user who will visit the customer but also the past reservation record information for at least several months are stored. The reservation ledger information can also be accessed from the restaurant terminal 300 .

The user information database 32 stores information about users who own the user terminals 200 and are users (members) of the restaurant information service through the portal site provided by the restaurant information providing server 100 . Specifically, the user information database 32 stores information such as a user ID, password, name, e-mail address (other message destination information), telephone number, address, age (group), gender, birthday, etc. I remember each time.

Furthermore, the user information database 32 stores face image data of each user and feature data for face authentication ( registration data). In addition to this, face image data with the mask on and data on the hairstyle of the user may be stored.

The store visit history information database 33 stores information about users who have visited restaurants and are identified based on the image captured by the entrance camera Cd and the image captured by the in-store camera Ci, and the order information (POS data) on the table T at which the user has arrived. Stores historical information. Specifically, the store visit history information database 33 includes, for each restaurant, visit date and time (time zone), user ID, table ID, POS data (for example, usage amount, order menu, information on privilege usage status, etc.), stay time, and so on. , frequency of visits, etc. are associated with each other and stored.

These databases are mutually referenced and used as necessary in the user-table linking process by the restaurant information providing server 100, which will be described later.

[Operation of restaurant information providing server]
Next, the operation of the restaurant information providing server 100 configured as above will be described. The operation is executed by cooperation between hardware such as the CPU 11 and the communication unit 19 of the restaurant information providing server 100 and software stored in the storage unit 18 . In the following description, for convenience, the CPU 11 is assumed to be the operating entity.

FIG. 4 is a flow chart showing the flow of user-table linking processing by the restaurant information providing server 100 .

As shown in the figure, the CPU 11 of the restaurant information providing server 100 first receives the captured image from the entrance camera Cd (step 41).

Subsequently, the CPU 11 determines whether or not a person's face has been detected in the image captured by the entrance camera Cd (step 42).

If it is determined that no face has been detected in the captured image (No in step 42), the CPU 11 returns to step 41 to receive a new captured image from the entrance camera Cd.

If it is determined that a face has been detected in the captured image (Yes in step 42), the CPU 11 extracts the face area in the captured image from the registration data registered in the user information database 32 (face image of each user). feature data) (step 43).

Subsequently, the CPU 11 determines whether or not the user in the captured image has been specified by the collation (step 44).

When determining that the user in the captured image could not be specified (No in step 44), the CPU 11 terminates the user/table linking process. In this case, the CPU 11 may record that a user whose face image is not registered has visited the store.

If it is determined that the user in the captured image has been identified (Yes in step 44), the CPU 11 stores the user ID of the identified user in the store visit history information database 33, and authenticates the user based on the captured image of the entrance camera Cd. The user with the user ID is excluded from the targets (step 45). That is, the CPU 11 excludes the registration data of the specified user from the registration data when matching the captured image of the entrance camera Cd with the registration data regarding the face images of the plurality of users. As a result, wasteful processing for matching the specified user's face image is omitted, and the load is reduced.

Subsequently, the CPU 11 receives the captured images from the plurality of in-store cameras Ci (step 46).

Subsequently, the CPU 11 collates the face area in the captured image of the in-store camera Ci with the registered data of the identified user (step 47).

Subsequently, the CPU 11 determines whether or not the specified user can be detected from the image captured by the in-store camera Ci (step 48).

If it is determined that the specified user could not be detected from the image captured by the in-store camera Ci (No in step 48), the CPU 11 returns to step 46 and receives a new image captured by the in-store camera Ci.

When it is determined that the specified user has been detected from the image captured by the in-store camera Ci (Yes in step 48), the CPU 11 specifies the table T where the user has arrived (or is about to arrive) (step 49). ). Specifically, the CPU 11 selects the number and characters of the table T at which the detected user is arriving (or about to arrive) from among the numbers and characters attached to each table T for identifying each table. The table T is specified by recognizing it from inside. Alternatively, the CPU 11 stores in advance information specifying each table T in the captured image according to the position in the captured image of the table T within the angle of view of each in-store camera Ci, and the user in the captured image is detected. The table T may be specified based on the position obtained.

In the process of specifying the table T, when the user simply passes by the vicinity of the table T existing on the way from the entrance to the table T at which the user finally arrives, the table T that the user has passed is the table T at which the user has arrived. In order to prevent the user from being identified, the CPU 11 may set a condition for the identification processing of the table T that the detected user remains in the same place for a predetermined time (eg, 5 seconds, 10 seconds, etc.).

Subsequently, the CPU 11 associates the user ID of the specified user, the table ID of the specified table T, and the POS data of the table T, and stores them in the visit history information database 33 (step 50). The CPU 11 may start the memory processing when the user places the first order in the table T as a trigger. POS data is updated each time an order is placed by a user. Here, the process of associating the user ID and table ID with the POS data is not essential and may be omitted. In other words, the CPU 11 may at least execute the process of associating the user ID of the specified user with the table ID of the table T at which the user has arrived.

Subsequently, the CPU 11 determines whether or not the specified user has left the store (step 51). Specifically, the CPU 11 captures a captured image (third captured image) by an exit camera (third camera; not shown) that is installed at the exit of the restaurant and is capable of capturing the face of the user leaving the restaurant substantially from the front. When the user is detected in , it may be determined that the user has left the store. The exit camera may also be used as the entrance camera Cd. Alternatively, the CPU 11 may determine that the user has left the store when it determines that the payment is completed in the POS data of the table T associated with the specified user.

When it is determined that the user has left the store (Yes in step 51), the CPU 11 includes the user who has left the store again in the authentication target by the captured image of the entrance camera Cd (step 52). That is, the CPU 11 adds the registration data of the user who has left the store to the registration data when matching the image captured by the entrance camera Cd with the registration data regarding the face images of the plurality of users. If it is determined that the user has not left the store (No in step 51), the determination of whether or not the user has left the store is repeated.

The CPU 11 repeats the above processing each time a (new) face is detected from the image captured by the entrance camera Cd.

[summary]
As described above, according to the present embodiment, the restaurant information providing server 100 identifies the user based on the captured image of the entrance camera Cd with high authentication accuracy, and transfers the identified information to the in-store camera Ci. Therefore, in order to identify the user alone in relation to the installation position, etc., the user is detected even from the captured image of the camera Ci in the store with low authentication accuracy, the table T used by the user is identified, and both are linked. be able to. Further, according to the present embodiment, the restaurant information providing server 100 links the authentication information of each camera, even if the captured images of the entrance camera Cd and the in-store camera Ci do not overlap each other. A table T can be specified. As a result, it is possible to link the POS data corresponding to the table T with the user, and the restaurant can provide information such as which user visited the store, what kind of menu they ordered, and how much money they spent. Attribute information can be collected and used for sales promotion activities and product/system development.

[Modification]
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present disclosure.

In the above-described embodiment, the restaurant information providing server 100 excludes the specified user from the user specified by the image captured by the entrance camera Cd from the time the user is specified until the time the user leaves the restaurant. In addition to or instead of this, the restaurant information providing server 100 displays images captured by the entrance camera Cd from the time the specified user leaves the restaurant until the restaurant closes on the day the user visits the restaurant. The specified user may be excluded from the user specified by. As a result, the restaurant information providing server 100 prevents the user from specifying processing by the entrance camera Cd until the end of the day's business, because the user who has visited the restaurant once does not usually return to the restaurant on the same day after leaving the restaurant. By excluding it, wasteful processing can be omitted and the load can be reduced.

Furthermore, in the above-described embodiment, the restaurant information providing server 100, during the period from when the identified user is identified until he/she leaves the restaurant, The specified users may be excluded from the specified users. As a result, the restaurant information providing server 100 does not allow a user who visits a certain restaurant to visit another restaurant at the same time. By excluding it from the specific processing by the entrance camera Cd, unnecessary processing can be omitted and the load can be reduced.

In the above-described embodiment, the restaurant information providing server 100 compares the image captured by the in-store camera Ci with the registration data of the face image of the user identified from the image captured by the entrance camera Cd, thereby confirming that the user has arrived. The table T that was located was specified. However, the user's face may not be detected depending on the angle of the user captured in the captured image of the in-store camera Ci. Therefore, the restaurant information providing server 100 compares the image captured by the in-store camera Ci with the registration data of the user's face image, and also compares the image other than the user's face included in the image captured by the entrance camera Cd with the image captured by the entrance camera Cd. The table T at which the identified user has arrived may be identified by collating images other than the user's face included in the captured image of the camera Ci. As a result, even if the restaurant information providing server 100 cannot detect the user's face from the image captured by the in-store camera Ci, the user's body parts other than the face (hairstyle, etc.), the user's clothes, belongings (bag, etc.), height, etc. It becomes possible to detect the user from such as.

In the above-described embodiment, the restaurant information providing server 100 selects from the image captured by the in-store camera, which is capable of capturing the area (passage, etc.) between the entrance of the restaurant and the table T, among the plurality of in-store cameras Ci. The table T at which the user has arrived may be specified by detecting the user who has arrived at the table and further detecting that the user has arrived at the table T. FIG. More specifically, immediately after the user is identified from the captured image of the entrance camera Cd, the restaurant information providing server 100 considers that the user is heading from the entrance toward the floor where the table T is provided in the store. At least for a predetermined period of time (for example, 30 seconds) from the identification of the user, the in-store camera Ci used for detecting the user in the store is limited to the in-store camera Ci installed between the entrance and the area where the table T is provided. Then, for example, when a user is detected by an in-store camera Ci provided in the aisle between the entrance and the table T, after that, the captured image of a plurality of in-store cameras Ci provided further inside the store sequentially detects the user. may be tracked. As a result, the restaurant information providing server 100 can quickly detect the user inside the restaurant by limiting the use of those cameras, among the plurality of in-store cameras Ci, that are highly likely to detect the specified user. Processing can be omitted and the load can be reduced.

In the above-described embodiment, an example is shown in which the table identification process based on the image captured by the in-store camera Ci is executed for one user identified from the image captured by the entrance camera Cd. However, when a plurality of users are captured in the captured image of the entrance camera Cd, or when a plurality of users visit the store in a short period of time, a plurality of users are identified from the captured image of the entrance camera Cd, and the table for one user is displayed. It is conceivable that table identification processing for other users may be required before the identification processing is completed. Therefore, the restaurant information providing server 100 identifies a plurality of users based on the captured image of the entrance camera Cd, and based on the captured image of the in-store camera Ci and the registration data of the identified users, When specifying the table T at which a plurality of users are seated, if the table T at which one of the plurality of users is seated can be specified, the in-store camera is used until the user leaves the store. The user may be excluded from the target of identification processing of the table T by the captured image of Ci (that is, the registration data of a user whose table has already been identified may be excluded from the registration data of a plurality of users to be collated). As a result, the restaurant information providing server 100, even when table identification processing based on the image captured by the in-store camera Ci is accumulated for a plurality of users regarding the user identified by the image captured by the entrance camera Cd, can be used to identify the image captured by the in-store camera Ci. If one of these users is detected, the registration data of that user is excluded from the verification target of the subsequent specific processing, thereby omitting unnecessary processing and reducing the load. can.

In the above embodiment, only one restaurant information providing server 100 is shown, but the processing executed by the restaurant information providing server 100 may be distributed and executed by a plurality of servers. For example, a separate server may be responsible only for matching the images captured by the entrance camera Cd and the in-store camera Ci with the registration data.

Further, in the above-described embodiment, the restaurant information providing server 100 on the cloud executes the linking process between the user and the table using the captured image in each restaurant, but each restaurant terminal 300 or provided separately from it The server of the restaurant may execute the linking process.

Among the inventions described in the claims of the present application, the invention described as "information processing method" automatically performs each step by at least one device such as a computer by information processing by software. It is not performed by humans using devices such as computers. That is, the "information processing method" is an information processing method using computer software, and is not a method of manipulating a computing tool such as a computer.

11 CPU
18... Storage unit 19... Communication unit 31... Restaurant information database 32... User information database 33... Visit history information database 100... Restaurant information providing server 200... User terminal 300... Restaurant terminal Cd... Entrance camera Ci... In-store camera

Claims (11)

Based on a first captured image captured by a first camera installed at the entrance of a restaurant and capable of capturing the faces of users visiting the restaurant from almost the front and from a short distance , and registered data on the faces of a plurality of users stored in advance. to identify the user who visited the store,
Based on the registered data of the specified user from a second captured image captured by a second camera that is installed in the restaurant and capable of capturing a bird's -eye view of the floor on which a plurality of tables are arranged in the restaurant , detecting the identified user, identifying the table at which the detected user has arrived from among the plurality of tables ,
storing user identification information identifying the identified user and table identification information identifying the identified table in association with each other;
An information processing system comprising a controller. The information processing system according to claim 1,
The information processing system, wherein the control unit associates and stores the user identification information of the specified user and the POS data of the specified table. The information processing system according to claim 1 or 2,
The information processing system, wherein the control unit excludes the identified user from the identification target of the user based on the first captured image from when the identified user is identified until he/she leaves the store. The information processing system according to claims 1 to 3,
The control unit identifies a plurality of users based on the first captured image, and determines that the plurality of users arrive based on the second captured image and the registration data of the identified plurality of users. When specifying each of the tables, if the table at which one of the users has arrived can be specified, the table is specified by the second captured image until the user leaves the store. An information processing system that excludes the user from the target of processing. The information processing system according to claim 3 or 4,
The control unit determines whether the specified user has left the restaurant based on a third captured image captured by a third camera that is installed at the exit of the restaurant and capable of capturing the face of the user leaving the restaurant from substantially the front. An information processing system that determines whether or not The information processing system according to claim 3 or 4,
The information processing system, wherein the control unit determines whether the specified user has left the store based on POS data in a table associated with the specified user. The information processing system according to any one of claims 1 to 6,
An information processing system in which an imaging range of the first camera and an imaging range of the second camera do not overlap. The information processing system according to any one of claims 1 to 7 ,
The control unit compares the second captured image with the registered data of the specified user, and compares an image other than the face of the user included in the first captured image with the second captured image. An information processing system that identifies the table at which the identified user is seated by collating with included images other than the face of the user. The information processing system according to any one of claims 1 to 8 ,
The control unit detects the specified user from a second captured image of a second camera capable of capturing an area between the table and the entrance, and specifies the table at which the specified user has arrived. Information processing system . Based on a first captured image captured by a first camera installed at the entrance of a restaurant and capable of capturing the faces of users visiting the restaurant from almost the front and from a short distance , and registered data on the faces of a plurality of users stored in advance. to identify the user who visited the store,
Based on the registered data of the specified user from a second captured image captured by a second camera that is installed in the restaurant and capable of capturing a bird's -eye view of the floor on which a plurality of tables are arranged in the restaurant , detecting the identified user, identifying the table at which the detected user has arrived from among the plurality of tables ,
An information processing method, wherein user identification information for identifying the identified user and table identification information for identifying the identified table are stored in association with each other. information processing equipment,
Based on a first captured image captured by a first camera installed at the entrance of a restaurant and capable of capturing the faces of users visiting the restaurant from almost the front and from a short distance , and registered data on the faces of a plurality of users stored in advance. a step of identifying the user who visited the store;
Based on the registered data of the specified user from a second captured image captured by a second camera that is installed in the restaurant and capable of capturing a bird's -eye view of the floor on which a plurality of tables are arranged in the restaurant , detecting the identified user and identifying a table at which the detected user has arrived from among the plurality of tables ;
and storing the user identification information identifying the identified user and the table identification information identifying the identified table in association with each other.

Images