JP2004133904A - System for collecting marketing data for restaurant - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for collecting marketing data for a restaurant that can improve contents of a set-type cuisine or a course-type one to be desirable (contributing to the sales or profits of restaurants) for a customer by grasping how each cuisine for composing the contents of the set-type cuisine or the course-type one is received by the customers. <P>SOLUTION: The system comprises a data analysis means for obtaining "a time required for eating/drinking by a customer, whether the customer has left half eaten (half drunken), or the amount of food that the customer has left half eaten" regarding each cuisine for composing the set-type cuisine or course-type one; and a database recording means for recording "a time required for eating/drinking by a customer, whether the customer has left half eaten, or the amount of food that the customer has left half eaten" in a database for each customer regarding each cuisine for composing the set-type cuisine or the course-type one, based on information from the data analysis means. <P>COPYRIGHT: (C)2004,JPO

Description

The present invention provides a first-type course-type dish provided in a restaurant, "a plurality of dishes and one or more beverages are provided in a plurality of stages at different times from each other," The second stage is that “a plurality of dishes, or a plurality of dishes and one or more beverages” are provided, and the second stage is “beverage only or desserts and beverages”. Type of course-type dishes "," set-type dishes in which "a plurality of dishes and one or more drinks" are provided at substantially the same time ", or individual dishes or contents that constitute the contents of" buffet dishes " For each of the beverages, how the customer feels (whether they feel delicious / preferred or not), that is, each of the dishes or beverages that make up the course, set, or buffet food is sold at the store. It relates to a system for collecting marketing data of how to or how much to contribute to profits.

Conventionally, there is a technique in which a camera captures an image of food, drink, or tableware provided by a restaurant and uses the image (see, for example, Patent Documents 1, 2, and 3). However, all of these technologies are merely image processing and image recognition technologies for discriminating and identifying types of single dishes such as sushi, or merely for the purpose of billing and settlement in a cafeteria. It is not intended to collect marketing data on individual dishes or beverages that constitute the contents of course-type dishes, set-type dishes, or buffet dishes provided by restaurants.

Conventionally, for dishes or beverages ordered by customers as "single items", such as one beer containing a mug, one serving of yakitori, one hamburger, one curry rice, one beef bowl, etc. Record and use as marketing data data such as the order quantity of each item, the day and time of the order for each item, and the attributes (eg, age group, gender, occupation, annual income, family composition) of the customer who ordered each item. A restaurant POS system is known which can be used. In the case of using such a restaurant POS system, "a set in which a plurality of dishes (including drinks such as soups) are provided on a dining table almost at the same time (almost at the same time) as they are or placed on a table or the like. "Type dish" or "course-type dish in which a plurality of dishes or beverages (such as soups and coffee after meals) are provided in a plurality of stages at different times from each other in a predetermined order" itself, It is possible to grasp the date and time and the attributes of the customer who ordered it.

However, even when such a restaurant POS system is used, conventionally, each of the individual dishes or beverages constituting the contents (breakdown) of the set-type dishes or the course-type dishes as described above is provided to the customer. How they are received, i.e., how much of the dishes or beverages the customer feels and which ones are not, i.e. how many of the dishes or beverages are in the store It was not possible to ascertain whether it was contributing to the sales or profits.

Further, conventionally, for each of the individual dishes or beverages constituting the course-type or set-type dishes, which are popular and unpopular with customers of what attributes, what seasons, days of the week, It was not possible to know whether it was popular or unpopular at the time of day and weather.

(4) Conventionally, it has not been possible to know which of the various dishes or beverages provided as “buffet dishes” are popular or unpopular with customers. Further, conventionally, among various dishes or beverages provided as “buffet dishes”, which dishes or beverages are popular and unpopular with customers of what attributes, what season, day of the week, It was not possible to know whether it was popular or unpopular at the time of day and weather.

From the point of view of the restaurant, for the set-type dishes or the course-type dishes as described above, for each of the dishes constituting the contents, how each is received by the customer, that is, the customer Or which of the beverages tastes better or which ones feel less delicious or lesser, i.e., which of the dishes or beverages contributes more to the store's sales or profits and which contributes more Without knowing whether or not the set-type dishes or the course-type dishes are more desirable for customers (those that can further contribute to store sales or profits), they cannot be improved.

Furthermore, for each of the individual dishes or beverages that make up the course-type or set-type dishes, which are popular and unpopular with customers of what attributes / customers, what season / day / time / Without knowing whether it is popular or unpopular at the time of the weather, the above-mentioned set-type or course-type dishes are more suitable for customers (customer segments) whose attributes the store aims at as a main target, each season, It cannot be improved to one that is more suitable for the day of the week, time zone, weather, store location characteristics, etc. (which can further contribute to store sales or profits).

In addition, regarding the “buffet cuisine”, which dishes or beverages among various dishes or beverages are popular and unpopular with customers, and further, which cuisines or beverages are popular with customers of what attributes and popular If it is not known whether or not it is popular and unpopular at what season, day of the week, time of day, and weather, various dishes or beverages provided as the buffet food are more preferable for customers (furthermore, It cannot be adapted to each season, day, time zone, weather, store location characteristics, etc.).
JP 2000-245609 A JP 2000-107006 A JP-A-5-266358

The present invention has been made in view of such problems of the related art, and each of the individual dishes or beverages constituting all or a part of the contents of a set-type dish or a course-type dish is provided to a customer. How it is received, i.e., which of the dishes or beverages the customer feels delicious / preferable, which ones are not delicious / preferable, and which Can be used to determine what attributes are popular and unpopular (in what season, day of the week, time of day, and weather), and can serve as a set-type or course-type dish And a marketing data collection system for a restaurant, which can improve the customer's service to something more favorable for customers (one that can contribute more to store sales or profits). An object of the present invention is to provide a fine way.

In addition, the present invention, among various dishes or beverages provided as "buffet dishes", which dishes or beverages are popular and unpopular with customers, and furthermore, which dishes or beverages have what attributes Knowing which customers are popular and what is not popular (in addition to what season, day of the week, time of day, and weather), various dishes or beverages offered as buffet dishes are more favorable to customers (more It is an object of the present invention to provide a restaurant marketing data collection system and method that can be improved to those that can contribute to store sales or profits.

<< (Content of the present invention) >> The content of the present invention is as follows.

1. When "a plurality of dishes and one or more beverages are provided in a plurality of stages and are provided in a plurality of stages, a first-type course-type dish" is provided to a customer in a restaurant, the customer of the course-type dish may During the course of eating or drinking, or when the eating and drinking is finished, for each of the dishes or beverages of each stage constituting the course-type dish, `` whether or not the customer left and eat, the amount of the customer left and eat, or Data analysis means for obtaining information of `` time required by the customer for eating and drinking '', based on the information obtained by the data analysis means, for each dish or beverage constituting the course-type dish, for each customer, A marketing database for recording "whether or not the customer has left or not eaten, the amount of the customer who left or missed the meal, or the time required for the customer to eat and drink". Chromatography data collection system.

2. "A plurality of dishes or a plurality of dishes and one or more beverages are provided as a first step, and a" beverage or dessert and a beverage "are provided as a second step. In the case where the dishes or beverages of the first stage or the second stage constituting the “type course-type dish” are provided to customers in a restaurant, each provided as the first stage or the second stage is provided. When the food or drink by the customer of the food or beverage is in progress or when the food or drink is finished, the customer is asked, "For each of the foods or beverages that constitute all or a part of the first stage or the second stage, Whether or not the food or drink is left or not, the amount of the food or drink left by the customer, the time required for the food or drink by the customer ", and / or Customer drinking of food or beverages Has been started, or a predetermined stage at which the eating and drinking by the customer regarding each dish or beverage in the group of dishes or beverages constituting all or a part of the first stage has been completed or near to the end has been reached. Data analysis means for obtaining information of the order, and, based on the information obtained by the data analysis means, for each of the dishes constituting the course-type dish, for each customer, `` whether or not the customer has left or not eaten The amount left by the customer eating and drinking, the time spent by the customer eating and drinking, and the eating and drinking by the customer with respect to each dish or beverage in the group of dishes or beverages that constitute all or a part of the first stage has been started. The order, or the predetermined stage at which the eating and drinking by the customer with respect to each of the dishes or beverages in the group of dishes or beverages constituting all or a part of the first stage is completed or near the end. Marketing data collection systems for restaurants, which comprises by a marketing database for recording order "was the.

3. When "set dishes" in which "a plurality of dishes and one or more beverages" are provided almost simultaneously, are provided to a customer in a restaurant, during or after the eating and drinking by a customer of the set dishes. When the eating and drinking is completed, the following message is displayed for each of the dishes or beverages constituting the whole or a part of the set-type dish, whether or not the customer has left or not eaten, the amount of the customer's left and right, Or "and / or" the order in which eating and drinking by the customer regarding each dish or beverage in the group of dishes or beverages that constitute all or part of the set dish, or The order in which the customer's eating and drinking for each dish or beverage in the group of dishes or beverages constituting whole or a part has been completed or has reached a predetermined stage near the end. Data analysis means and, based on the information obtained by the data analysis means, for each of the dishes or beverages constituting the set-type dish, for each customer, `` whether or not the customer has left or not eaten, The amount left, the time required by the customer for eating and drinking, the order in which eating and drinking by the customer for each dish or beverage in the group of dishes or beverages constituting all or a part of the set-type dish, or the set A marketing database for recording the order in which eating and drinking by the customer with respect to each dish or beverage in the group of dishes or beverages constituting all or a part of the type dish has been completed or has reached a predetermined stage near the completion of the food or beverage, A marketing data collection system for a restaurant, comprising:

4. "Buffet food, in which the customer places various dishes or drinks arranged in a predetermined place in the restaurant on his own table according to his preference and places it on his own table," Data analysis means for obtaining information of "the type and / or number of dishes or beverages placed on their own dining table by the customer's own dishes, when provided to customers in the store, On the basis of the information obtained by the data analysis means, for each buffet dish, for each customer, "type and / or number of dishes or drinks placed by customers on their own dining table in their own dishes And a marketing database for recording the marketing data collection system.

5. In the above-mentioned 1, 2 or 3, the data analysis means, for each dish or beverage provided on the dining table of the restaurant seat floor of the restaurant, "whether or not the eating and drinking by the customer has been completed or not. A termination data for judging whether or not a near predetermined stage has been reached.

6. In the above 1, 2 or 3, the data analysis means determines whether or not each dish or beverage has been provided to the customer for each dish or beverage provided on the dining table on the seating floor of the restaurant or whether the customer Means for determining whether or not the customer has been provided with each dish or beverage or means for determining when the customer has started eating and drinking, and whether the customer has finished eating or has reached a predetermined stage near the end. A means for determining whether the customer has finished eating or drinking or a time point when the customer has reached a predetermined stage near the end of eating and drinking. A marketing data collection system for a restaurant, comprising:

7. In any one of the above items 1 to 6, the marketing database stores the information obtained by the data analysis means as “attribute of each customer, month and day when providing the eating and drinking service, day of the week when providing the eating and drinking service, The time or time zone at the time of providing the eating and drinking service, the season at the time of providing the eating and drinking service, the weather at the time of providing the eating and drinking service, the temperature at the time of providing the eating and drinking service, or the location characteristics of the store that provides the eating and drinking service. And a marketing data collection system for restaurants.

8. In the above-mentioned 5, the video game device further includes video acquisition means for acquiring a video of each dish or beverage on the dining table, and the end determination means uses the video information from the video acquisition means to set a dining table on a customer floor of a restaurant. It is determined whether the eating and drinking by the customer has been completed for each dish or beverage provided on the table or whether the eating and drinking by the customer has reached a predetermined stage near the end (for example, each dish or beverage on the dining table). Is almost gone or when the state of each dish or beverage on the table does not fluctuate for a predetermined time or more, it is determined that the food or drink by the customer has reached or reached a predetermined stage of eating or drinking by the customer. ) A marketing data collection system for a restaurant, characterized in that:

9. In the above 5, the method further includes infrared data acquisition means for acquiring infrared data emitted from each dish or beverage on the dining table, wherein the end determination means is based on the infrared data from the infrared data acquisition means. It is determined whether the eating and drinking by the customer has been completed for each dish or beverage provided on the dining table on the customer's seat floor or whether or not the eating and drinking by the customer has reached a predetermined stage near the end (for example, the dining table). When the emission of infrared rays from each of the above dishes or beverages has almost disappeared, when the temperature of each of the dishes or beverages on the table has dropped below a predetermined temperature for a predetermined time or more, or each of the dishes on the table has Or, when the temperature of the beverage has decreased by at least a predetermined temperature from the temperature at the time of provision, and a predetermined time or more has elapsed, the food or drink of the food or beverage by the customer Ryo or at a determined) as to have reached a predetermined stage close to the end, marketing data collection systems for restaurants, characterized in that.

10. In the above item 5, the apparatus includes shape data acquisition means for acquiring shape data of each dish or beverage on the dining table, and the end determination means includes a restaurant based on the shape data acquired by the shape data acquisition means. It is determined whether the eating and drinking by the customer has been completed for each dish or beverage provided on the dining table of the customer seat or whether the eating and drinking by the customer has reached a predetermined stage near the end (for example, the table on the dining table). When the food or beverage of the above is almost exhausted or when the shape of each food or beverage on the table does not fluctuate for a predetermined time or more, the food or drink by the customer for the food or beverage reaches or reaches a predetermined stage near the end. A marketing data collection system for a restaurant.

11. In the above 5, the “shape, position, and / or orientation of each dish such as a plate, a cup, a bowl, a fork, a knife, a spoon, and chopsticks” provided on the table together with each dish or beverage on the table is described. Tableware data acquisition means for acquiring tableware data to be shown, wherein the end determination means is provided on a dining table on the customer seat floor of a restaurant based on the tableware data acquired by the tableware data acquisition means. It is determined whether the eating and drinking by the customer has been completed for each dish or beverage or whether or not the eating and drinking by the customer has reached a predetermined stage near the end (for example, “dish, cup, teacup, fork, knife, If the shape, position, and / or orientation of each tableware such as spoons and chopsticks does not fluctuate for more than a predetermined time, eating and drinking by the customer for the dish or beverage is completed. Is determined to have reached a predetermined stage close to the end.For example, when "a state where forks and knives are arranged in parallel with each other" continues for a predetermined time or more on the dining table, Is determined that the eating and drinking by the customer has reached or reached a predetermined stage near the end).

12. In the above item 4, the data analysis means may use the image or shape data of each dish or beverage on the dining table to indicate, " A marketing data collection system for a restaurant that provides a buffet dish, which seeks information of “type and / or number”.

According to the present invention, for each dish or beverage that constitutes all or a part of the contents of the set-type dish or the course-type dishes of the first or second type, it is determined whether or not the customer has left or not Records the amount of food left and / or eaten and / or the time spent by the customer in a database for each customer. That is, whether the customer feels delicious or not delicious in each of the dishes or beverages that make up the set dish or the course dish, or in each of the dishes or beverages. The store can know with high accuracy which of them contributes to the sales or profit of the store and which does not contribute to the store. Preferred for more customers to "contents" of the type of cooking makes it possible to improve the (more things that can contribute to sales or profits of the store).

Further, according to the present invention, regarding the first stage of the second type of course-type dishes or the set-type dishes, the first stage of the second type of course-type dishes or the entire contents of the set-type dishes or Within each group of dishes or beverages that form a part, the order in which the customer has started eating and drinking for each dish or beverage in the group, and / or the first stage of the second type of course-type cuisine or Within a group of dishes or beverages that make up all or a part of a set-type dish, the order in which the customer has finished eating or has reached a predetermined stage near the end of eating and drinking for each dish or beverage in that group " The data is recorded in a database for each customer. Therefore, according to the present invention, how each of the dishes or beverages constituting the contents is received by the customer, that is, the first step or the set-type dish of the second-type course-type dish is constituted. Which of the dishes or beverages the customer feels are delicious and which are not, i.e., which of the dishes or beverages contributes more to the store's sales or profits and which contributes more The store can determine whether or not the contents of the first stage of the second type of course-type dishes and the contents of the set-type dishes are more favorable to the customer (more store sales). Or those that can contribute to profits).

Further, in the present invention, the information indicating how the customer receives each of the dishes or beverages constituting the contents of the course-type or set-type dishes is attributed to each customer, weather or temperature at the time of providing the service. , The date and time when the service is provided, the day of the week when the service is provided, the time when the service is provided, the season when the service is provided, the location characteristics of each store, etc. are recorded in the database. What kind of customer (customer segment), what day of the week, season, time zone, weather, or store location characteristics, what kind of content or type of food or It becomes possible to grasp with high accuracy whether the beverage contributes to sales and profits. Therefore, the store side, according to the location characteristics of each store, the attributes (customer layer) of the customer targeted by each store, each day of the week, each season, each time zone, or the weather or temperature at that date and time, It becomes possible to improve each dish or beverage that constitutes the contents of the set-type or course-type dish to an optimal one that can make a maximum contribution to increasing sales and profit of the store.

Furthermore, in the present invention, when a restaurant provides a buffet dish, the customer can select a desired number of dishes or drinks from various dishes or drinks provided as buffet dishes by a desired number. When the customer places the dish or drink on his own table, and recognizes the type and number of dishes or drinks that the customer has placed on the table, the customer recognizes the type and number. It is recorded in the database. Further, in the present invention, the information on the type and the number of the recognized dishes or beverages is further added to the attribute, date, time, time zone, day of the week, season, weather (including temperature), or store location of each customer. It can be recorded in association with attributes. Therefore, the store side knows with high accuracy what kind of food or drink many customers prefer, that is, what kind of food or drink contributes more to sales and profit. In addition, various dishes or beverages included in a buffet dish can be improved to further contribute to sales and profits. In addition, the store should determine what type of food or beverage should be used when targeting the customer of what attribute, and what day, season, time zone, weather, or store location characteristics. It is possible to grasp with high accuracy whether or not contributes to sales and profit, and to improve various dishes or beverages that are the contents of the buffet dish.

Hereinafter, the best mode of the present invention will be described with reference to the following examples.

FIG. 1 is a schematic view for explaining Embodiment 1 of the present invention. In FIG. 1, reference numeral 1 denotes a customer who uses a restaurant service, 2 denotes a dining table on which dishes provided to the customer 1 are placed, and a dining table (table) installed on the floor of a customer seat; The dishes to be provided (the dishes constituting the set-type dishes or the course-type dishes), 4 are dishes (plates) on which the dishes 3 to be served on the dining table 2 are placed, 21 is a mug containing soup, and 5 is This is a camera for imaging “dish 3, tableware 4, mug 21, and soup in mug 21” from above or obliquely above table 2.

In the first embodiment, the imaging range of the camera 5 is limited to only the table 2. That is, the camera 5 is set so that its imaging range does not capture images other than the hands in the body of the customer 1 handling the tableware, and in particular, does not capture the face of the customer 1 that leads to the privacy information of the customer 1. Have been.

In addition, the camera 5 is provided with an infrared camera (for detecting infrared rays emitted from the dish 3) by, for example, “attaching a“ filter that transmits only infrared rays ”inside or outside the objective lens of the camera 5”. (A type of infrared detecting means). The camera 5 is controlled by, for example, a “microcomputer connected to an operation switch of the camera 5” (not shown), and the filter is mounted inside or outside the objective lens (acting as an infrared camera). ) And when it is not mounted (when it operates as a normal visible light camera), it can be alternately switched at a predetermined timing.

In FIG. 1, reference numeral 6 denotes a control unit (CPU = central processing unit) provided in the kitchen of the restaurant, which controls the camera 5 and controls the camera 5 (both a visible light camera and an infrared camera). When a camera that plays a role) starts imaging or infrared detection, the information (imaging information from the camera 5 as a visible light camera and infrared detection result information from the camera 5 as an infrared camera) is transmitted in real time or at a predetermined time. It is a control unit for executing a program for taking in every time (for example, every few seconds).

In the first embodiment, the kitchen in which the control unit 6 is placed is arranged at a position where the dining table 2 on the floor of the customer's seat is completely invisible or hardly directly visible. .

In FIG. 1, reference numeral 7 denotes all kinds of dishes (single dishes, single drinks, set dishes, course dishes, and various dishes or drinks provided as buffet dishes, etc.) included in the menu provided by the restaurant. (Including information such as images when serving typical and standard examples, serving shapes, serving colors, serving temperatures, and serving weights) for each dish or beverage This is a cooking database (comprising a hard disk or the like) to be recorded, which is a cooking database read and written by the control unit 6.

In FIG. 1, reference numeral 8 denotes order contents for each dining table for inputting the order contents of dishes (including set-type dishes, course-type dishes, etc.) for each dining table 2 (= each customer) to the control unit 6. An input unit (configured with a keyboard or the like) 9 is a table-specific order for recording the order details for each table 2 (= each customer) input from the table-specific order content input unit 8. It is a content recording unit (comprising a hard disk or the like). In the first embodiment, each table-specific order content input unit 8 is configured by a portable information device carried by a customer service clerk. The order content input using this portable information device is: The portable information device wirelessly transmits the information to the control unit 6, and the control unit 6 inputs the information to each table-specific order content recording unit 9.

In FIG. 1, reference numeral 10 denotes a display for displaying predetermined character / diagram or image information under the control of the control unit 6, and reference numeral 11 outputs predetermined voice or sound information under the control of the control unit 6. A speaker 12 for printing out a paper cooking instruction slip or a serving instruction slip addressed to a cook or other clerk in the kitchen under the control of the control unit 6; Is a cook (cook, chef) working in the country.

In FIG. 1, reference numeral 15 denotes a date and time data generation unit (electronic calendar and clock unit) for constantly generating current date, day of the week, and time data.

In FIG. 1, reference numeral 14 denotes video data (and / or infrared data) from the camera (also serving as an infrared camera) 5 sent via the control unit 6, data from the food database 7 (each food or Based on each information such as the shape, color, temperature, etc. of typical / standard examples of beverages) and the current date / time data from the date / time data generator (timer) 15, the "set-type dish" Or, for each dish or beverage that constitutes a course-type dish, whether or not eating and drinking by the customer has been started, and whether or not eating and drinking by the customer has ended or has reached a predetermined stage near the end. This is a determination analysis unit for performing the analysis.

That is, based on the information, the determination analysis unit 14 respectively, for the respective dishes or beverages, respectively, for example, when the respective dishes or beverages are placed on the dining table 2 or on the dining table 2 When the shape of each dish or beverage fluctuates (when the area of the plane of each dish or the height and volume of each beverage decreases), it is determined that eating and drinking by the customer for each dish or beverage has been started, respectively. Record the date and time data.

In addition, the determination analysis unit 14 determines, for each of the above-mentioned dishes or beverages, for example, (a) that each of the above-mentioned dishes or beverages is mostly left and eaten based on each information from the camera 5 and the dish database 7. When there is no state, (b) when the state in which each of the dishes or beverages is not deformed or reduced has passed for a predetermined time or more, and (c) each of the dishes or beverages should be eaten or drink in a state where it is originally warm. However, when the temperature of each of the dishes or beverages has dropped below a predetermined temperature (for example, a temperature close to room temperature) for a predetermined time or more, or (d) the eating or drinking of each of the dishes or beverages is originally warm. However, when the temperature of each dish or beverage has fallen by more than a predetermined temperature (for example, 30 degrees) or more than the temperature at the time of providing, and a given time or more has elapsed, the dish or beverage is not attached to the dish or beverage. With food by the customer performs a termination determination that "has reached a predetermined stage close to the finished or finished", and records the date and time data when the the end determination. Note that a specific determination method regarding the end of eating and drinking by the determination analysis unit 14 will be described later with reference to FIGS.

In addition, the determination analyzing unit 14 determines the time of the start of eating and drinking by the customer for each dish or beverage, the time of the end determination, and the camera (also serving as an infrared camera) 5 transmitted via the control unit 6. Image data (and / or infrared data), data from the dish database 7 (data such as shapes, colors, and temperatures of typical and standard examples of each dish or beverage), and the date and time data generator 15 based on each piece of information such as the current date and time data, "for each dish or beverage constituting the set-type dish or the course-type dish, the time required for each customer to eat and drink, ) Or not, and the amount left over by each customer (left over).

The determination analysis unit 14 states that “each customer or each of the dishes or beverages constituting the set-type dish or the first stage of the second type course-type dish is required for eating and drinking. Of each customer, whether or not each customer left over (drinked), and the amount of each customer left over (drinked) "are analyzed by (a) the customer's eating and drinking Although it can be performed at the time when the termination is determined, as a simpler method, (b) when the customer stands to leave the store, (c) when the customer pays for the food and drink service, Alternatively, (d) the calculation can be performed at the time when the customer instructs the service clerk to calculate (account) the charge for the food service.

As described above, the determination analysis unit 14 transmits video data (and / or infrared data) from the camera (also serving as an infrared camera) 5 transmitted via the control unit 6 and data (and / or infrared data) from the cooking database 7. Table 2 based on each information such as the shape, color, temperature, etc. of typical / standard examples of each dish or beverage) and the current date / time data from the date / time data generator (timer) 15. For each of the above dishes or beverages, the determination that the eating and drinking by the customer has started and the recording of the date and time data at the time of the determination, the determination that the eating and drinking by the customer has reached or reached a predetermined stage near the end and the time of the determination Records date and time data.

The determination analysis unit 14 is configured such that the customer starts eating and drinking for each dish or beverage in a group of dishes or beverages that constitute all or a part of the first stage of the second type of course-style dishes or all or part of the set-type dishes. The time when it is determined that the customer has eaten and drinks, and the customer eats and drinks each of the dishes or beverages in the group of dishes or beverages that constitute all or a part of the first stage or the set-type dishes of the second type of course-type dishes On the basis of the data at the time when it is determined that the set-type dish has been completed or the predetermined stage near the end has been reached. For each group of dishes or beverages that make up all or a part, the order in which the customers began eating or drinking in the dishes or beverages in that group, or those groups The order in which the eating and drinking by each customer in a dish or beverage is finished or has reached a predetermined stage near the end (or the order in which each customer has finished or almost finished eating or drinking until there is no or little leftover food or drink) ) "Is analyzed.

Note that, here, the determination analysis unit 14 analyzes “all or a part of each dish or beverage that constitutes all or a part of the set-type dish or the first stage of the second-type course-type dish. The order in which eating and drinking by each customer for each dish or beverage in the group of dishes or beverages to be constituted was started, or each dish or beverage constituting the whole or a part of the set dish or the second type. The order in which the eating and drinking by each customer for each dish or beverage in the group of dishes or beverages constituting all or a part of the first stage of the course-type dish is completed or has reached a predetermined stage near the end ”; As in the first stage of the set-type dishes or the course-type dishes of the second type, “a plurality of dishes or one or a plurality of beverages” are provided to a customer almost simultaneously (at once, together). Place (Different from the case of the first type of course-type dishes in which a plurality of dishes or beverages are provided in a plurality of stages at different times from each other) (1) Dishes provided almost at the same time as described above Or information within a group of beverages that indicates in which order the customer has begun eating (drinking) each dish or beverage, i.e., which dish or beverage in each dish or beverage is first What to eat (drink), what to eat (drink), what to eat (drink), what to eat last (drink), and / or (2) as described above Information indicating the order in which the customer finished eating (drinking) each dish or beverage in the group of dishes or beverages provided at substantially the same time, that is, First finish eating (drinking) a throat dish or beverage What finished eating (drinking finished with) Next (finished drink) what to eat after what (finished drink) was finished eating the last one, is that information, of which indicates the order of the.

Therefore, as described above, in the group of dishes or beverages provided almost simultaneously as described above, the order in which eating and drinking by each customer is started, or the eating and drinking by each customer in all dishes or beverages is ended. Or the order in which a predetermined stage near the end is reached) is analyzed when the first stage of the set-type dish or the second type of course-type dish is provided, and the first type Is not analyzed when the course-type dishes are provided (because, in the case of the first-type course-type dishes, the order in which the customer starts and ends eating and drinking is predetermined by the store side. ).

As described above, in the system of FIG. 1, the determination analysis unit 14 provides each of the dishes or beverages that constitute the first stage of the set-type dishes or the second-type course-type dishes (provided almost simultaneously to customers). Food or beverage group), the time required for each customer to eat and drink, whether or not each customer left over (drinked), the amount each customer left over (drank), The information that indicates the order in which eating and drinking is started or the order in which eating and drinking by each customer is completed or reaches a predetermined stage near the end is analyzed.

In FIG. 1, reference numeral 16 denotes a customer attribute generation unit for acquiring and recording attribute information (gender, age group, occupation, family structure, annual income, etc.) of each customer 2 and outputting it to the outside; 19 is a weather data generation unit for acquiring weather information (including temperature information) and outputting the information to the outside, and 19 is a location characteristic (location attribute of a restaurant in a chain of restaurants). For example, a business location, a shopping district, a student district, a residential district, an urban district, a suburb, a countryside, a snowy country, a tropics, etc. is there.

In the first embodiment, for example, the customer attribute generating unit 16 wirelessly receives the customer attribute information input by the table-specific order content input unit 8 from the table-specific order content input unit 8. (Refer to the data transmission line 8a in FIG. 1). The received attribute information is once recorded in itself, and when there is an output request from the outside, the attribute information is externally (for example, the judgment analysis unit 14 or the database management unit 20 described later). ). The input of customer attribute information (gender, age group, etc.) to each dining table order content input section 8 is made up of each dining table order content input section 8 (portable information device carried by a waiter). When the waiter inputs the order contents from the customer for each dining table (by the identification number of each dining table (that is, the identification number of each customer)), the waiter then enters the keyboard of the order content input section 8 for each dining table. Done using

In the first embodiment, for example, when the customer presents a membership card such as a loyalty card to the store at a cash register (payment counter) at the time of reception or checkout, Using the member ID (identification information) recorded on the card as a key, a member database (not shown; a member database recorded on a server of a center connected to the customer attribute generating unit 16 via the Internet) may be accessed. To obtain the attribute information of the member.

Further, the weather data generating unit 17 fetches weather information (including temperature and humidity information) of the place where the restaurant is located from the server of the weather information providing service company at predetermined time intervals via the Internet and records it therein. In advance, when there is an output request from the outside, the information may be output to the outside (the determination analysis unit 14 or the database management unit 20 described later).

Also, in FIG. 1, 20 indicates that “each customer required food and drink for each set of dishes or beverages constituting the set-type dish or the first and second types of course-type dishes” from the determination analysis unit 14. Information indicating the time, whether or not each customer has left over (drinked), the amount of each customer left over (drinked), and “the set-type dish and the second type of course-type dish” The order in which eating and drinking by each customer is started for each dish or beverage in a group of all or some dishes or beverages constituting the first stage of the above, or each customer for each dish or beverage in the same group Indicating the order in which eating and drinking are completed or the order of reaching a predetermined stage near the end ”, the current date and time data from the date and time data generation unit (time counting unit) 15 sent via the determination analysis unit 14, Judgment analysis unit 14 The customer attribute data from the customer attribute generator 16 sent from the server, the weather information sent from the weather data generator 18, and the store location attribute information sent from the store location attribute generator 19. A database management unit for recording and managing in the restaurant marketing database 17 while associating.

In the first embodiment, according to the marketing database 17, the shop side (1) how much time the customer spends on eating or drinking each of the dishes or beverages constituting the set-type dishes and the course-type dishes. (Cooking and eating over a long period of time causes a decrease in the turnover rate of the customer and a decrease in the profit rate.) (2) Whether the customer has left behind (drinks) and The amount of food left over (drinked) (the food left uneaten is considered to be unsavory from the customer's point of view, causing a decrease in sales. ) Can be known.

Further, in the first embodiment, the marketing database 17 allows the store to determine each dish or beverage in the group of dishes or beverages that constitute the first stage of the set-type dish and the second type of course-type dish. And (3) the order in which the customer started eating and drinking and the order in which the customer finished eating or eating or reached a predetermined stage near the end, respectively. In addition, a dish or beverage that has finished eating and drinking sooner to the point where there is almost no food or drink left is perceived from the customer's point of view as being delicious and having a higher contribution to sales, and conversely, more. Customers who started eating and drinking late, or who finished eating or eating late or left a large amount of food and beverages, were perceived to be less delicious from the customer's perspective, and contributed to sales. Is low).

Moreover, in the first embodiment, the store side provides the above information (1), (2) and (3) in accordance with the attributes of each customer and the service according to the location characteristics of each store. It can be known according to the day of the week, time zone, season, and / or according to the weather and temperature at the time of providing the service. That is, for example, the store side feels how many young women in their twenties (or men in their thirties, high school students, elderly people with an annual income of 5 million yen or more, for example) feel about each of the above-mentioned dishes or beverages. Can be known. In addition, for example, in a business district, in a business district, on a Friday night, in winter, or in fine weather, it is possible to know whether or not each of the dishes or beverages is popular with customers.

In FIG. 1, the control unit 6, the cooking database 7, the determination analysis unit 14, the date / time data generation unit 15, the customer attribute generation unit 16, the weather data generation unit 18, the store location attribute generation unit 19, the database management unit 20, the marketing The database 17, the table-specific order content input unit 8, the table-specific order content recording unit 9, the display 10, the speaker 11, and the printer 12 are composed of, for example, a personal computer system installed in a restaurant kitchen. ing.

Next, the operation of the first embodiment will be described. Now, suppose that the customer 1 who has come to the restaurant has ordered a first type of course-type cuisine (for example, a full course of French cuisine). Then, the waiter associates the data of the order content with the identification number of the dining table at which the customer is seated (this is also used as the identification number of the customer at the time of providing the service) and associates the data with the portable information terminal (PDA). ) From a keyboard or the like, and the information is wirelessly transmitted to the control unit 6 by short-range wireless communication means such as infrared communication or Bluetooth (in this case, the portable information terminal is connected to each table in FIG. 1). It becomes another order content input section 8).

Also, at this time, the waiter also inputs customer attribute information such as the gender and age group of the customer sitting at the dining table in the portable information terminal in association with the dining table identification number, and inputs the information to the customer attribute. It is transmitted to the control unit 6 by a short-distance wireless communication means such as infrared communication or Bluetooth to the generation unit 15 (see 8a in FIG. 1).

The information of the order content input by the waiter is displayed on the display 10 by the control unit 6 together with each table identification number (= customer identification number) (or a paper cooking instruction slip addressed to the cook 13). Is printed out from the printer 12). The cook 13 looks at the screen of the display 10 (or looks at the printed cooking instruction slip) and cooks the first set meal (hors d'oeuvre) ordered by the customer 1, Give it to the waiter. The waiter serves this first dish on dining table 2.

In the first embodiment, when receiving the order content information from the table-specific order content input unit 8, the control unit 6 controls the camera 5 to start imaging the upper surface of the dining table 2. Immediately thereafter (for example, one second or several seconds later), the control unit 6 operates the camera 5 as an infrared camera (the control unit 6 attaches the “filter that transmits only infrared light” to the camera 5). Then, the camera 5 is operated as an infrared camera) to detect infrared rays (heat rays) emitted from the dishes 3 on the dining table 2 and the soup in the mug 21.

In the present embodiment, far-infrared rays (or near-infrared rays emitted from the sensor side to the surroundings) are reflected by the customer, and the dishes / beverages installed above the dining table 2 or heat rays emitted from the customer (human body) are reflected by the customer. When a detection signal from an infrared sensor for detecting the reflected wave that has returned) is transmitted to the control unit 6, the control unit 6 controls the camera 5 to start imaging the upper surface of the dining table 2. You may do so.

Further, in the present invention, a human sensor composed of a far-infrared sensor or the like is attached to the camera 5 itself, and when a customer (human body) sits on the dining table 2, a signal from the human sensor is automatically used. The camera 5 may be programmed to start imaging as a visible light camera and detection of infrared light as an infrared camera. That is, in the present invention, the camera 5 itself autonomously starts imaging as a visible light camera and detection of infrared rays from the dish and soup as an infrared camera without control by the control unit 6. You may do so.

The video information captured by the camera 5 under the control of the control unit 6 is taken into the control unit 6 at a predetermined interval (for example, every few seconds) almost in real time (or after a lapse of a predetermined time). This video information is supplied to the determination analysis unit 14 via the control unit 6 as needed. The determination analysis unit 14 analyzes the progress of eating and drinking by the customer based on the video information and the date and time data from the date and time data generation unit 15, and analyzes each of the dishes or beverages constituting the course-type dishes or the set-type dishes. It is determined whether the eating and drinking by the customer is almost completed.

In addition, the determination analysis unit 14 (including means for executing image processing / image recognition software) determines a start of eating and drinking by a customer for each of the dishes or beverages, a determination of ending the same, the video information, Based on the current date and time data (time data at the time of each of the determinations) from the date and time data generation unit 15, the following information is provided for each of the dishes or beverages constituting the set or course dish: (1) Set dish or course dish The time of eating and drinking by the customer of each dish or beverage constituting the dish (how long the customer finished eating each dish or beverage) is analyzed. That is, based on the video information and the date and time data from the date and time data generation unit 15, the determination analysis unit 14 determines (a) the time at which the customer started eating and drinking for each dish or beverage, and (b) the customer Calculates the time at which the food or drink ends or reaches a predetermined stage near the end, and from these (a) and (b), (c) information on the time required by the customer for the food or drink for each dish or beverage is obtained. Ask.

In addition, the determination analysis unit 14 determines (2) the amount of leftovers (leftovers) of each dish or beverage based on the video information and the date / time data from the date / time data generator 15 (this “ When the “amount” is zero, it is understood that there was no leftover after eating or drinking).

In addition, the determination analysis unit 14 determines, based on the video information and the date and time data from the date and time data generation unit 15, (a) the time at which the customer started eating and drinking for each dish or beverage, and (b) the customer The time at which the eating or drinking ends or reaches a predetermined stage close to the end is obtained, and based on these information, when the first stage of the set-type dish and the second type of course dish is provided, they are configured. (3) the order in which the customer started eating and drinking each dish or beverage, and the predetermined stage at which the customer finished or nearly finished eating and drinking each dish or beverage. The order of reaching is analyzed and determined.

Then, the determination analysis unit 14 transmits the analysis result to the database management unit 20 in association with the customer attribute information and the date and time data (information on the date and time when each dish or beverage was provided), and transmits the result to the marketing database. Record at 17.

In the present invention, the “judgment analysis unit 14, store location characteristic generation unit 19, database management unit 20, and marketing database 17 (and, in some cases, the weather data generation unit 18, the customer attribute generation unit 16)) does not need to exist inside each store. For example, a server of a remote management center (for example, the headquarters of a company that develops a chain of multiple stores) connected to the control unit 6 via the Internet may be used. It may be configured. In particular, when a restaurant chain is developed, it is convenient to install the store location attribute generation unit 19, the database management unit 20, the marketing database 17, and the like on the management center side.

In the first embodiment, the determination analysis unit 14 determines based on the video information (and / or infrared information) from the camera 5 and the result of the determination of the start and end of the eating and drinking of each dish or beverage by the customer. The time from the start of eating and drinking by the customer for each of the dishes or beverages constituting the set-type dishes or the course-type dishes to “end (including the case where there is leftovers)” is measured. I have.

For example, based on an image from the camera 5, the determination analysis unit 14 may determine the amount, shape, or color of each dish or beverage that constitutes the first stage of the set dish or the second type of course dish. For example, the area, shape, and color of a dish or beverage on a plate may be the standard amount, shape, or color of each dish or beverage from the cuisine database 7 (eg, the flatness of a dish or beverage on a plate). (Area, shape, color), and if any deformation, reduction, or discoloration occurs, it is determined that the customer has started eating and drinking for the dish or beverage from that point . In addition, the determination analysis unit 14 determines the amount of each dish or beverage that constitutes the first stage of the set-type dish or the second type of course-type dish based on the infrared data (infrared image) from the camera 5. Or the shape (e.g., the area or shape of the dish or beverage on the plate) is the standard amount or shape of each dish or beverage from the dish database 7 (e.g., the area of the dish or beverage on the plate) , Shape), it is determined that the customer has started eating and drinking for the dish or beverage from that point on (from the point of view of the infrared image, the part of the dish or beverage is eaten or consumed by the customer). If so, from that point on, the infrared image of the dish or beverage will be deformed or reduced) . (Note that, with respect to the first type of course-type dishes , based on the image from the camera 5, the point in time at which a predetermined dish or beverage is provided on the dining table is determined by the customer for the dish or beverage at that stage. Is determined to be the start of eating and drinking.)

In addition, the determination analysis unit 14 determines, based on the video from the camera 5 and the infrared data, the amount of each dish or beverage (or the flat area on the plate) constituting the set-type dish or the course-type dish. When it is almost exhausted, or when it is left uneaten (drinked) (for example, a state where only a part is eaten for a longer time than a predetermined reference time, and the When a predetermined time or more has elapsed after the temperature of the dish has dropped to room temperature), it is determined at that time that the customer has finished eating and drinking the dish or beverage.

In the first embodiment, among the determinations by the determination analysis unit 14, the determination at the time when the customer starts eating and drinking for each of the dishes or beverages is relatively easy. Since the determination of the customer's eating and drinking at “the point when the customer has finished or reached a predetermined stage near the end” is relatively complicated, the determination analysis unit 14 in the first embodiment determines whether the customer has finished eating or drinking. The method of performing the operation will be described with reference to FIGS.

(A) First, a first method for determining the end of the eating and drinking will be described with reference to FIG. FIG. 2 shows an example of video information (video showing the table 2 in FIG. 1) from the camera 5. In the first method, the determination analysis unit 14 determines the actual “remaining amount” of the dish 3 in the tableware 4 (or the tableware 4) based on the video information (see FIG. 2) from the camera 5. The actual ratio of the area or volume (and color) occupied by the actual dish 3 in the area or volume (and color) of the entire tableware 4) is a known image processing program. This "remaining amount" (or actual ratio) is determined by the predetermined reference amount (or the predetermined area or volume (and color) of the dish) and the area or volume of the entire tableware. (And a reference ratio indicating the ratio of the occupants in the (color)), it is determined that "the eating and drinking of the above-mentioned dish (for example, hors d'oeuvre) has reached a predetermined stage near the end". I have.

That is, in the first method, the determination analysis unit 14 determines from the cooking database 7 a predetermined reference amount (or between the cooking and the tableware in which the cooking is placed) corresponding to the cooking corresponding to the order contents of the dining table 2. The reference amount (or reference ratio indicating the mutual relationship between the respective areas or volumes) is taken out, and the reference amount (or reference ratio) and the actual “remaining amount” (or the actual ratio) obtained from the image from the camera 5 ) To determine whether or not the food and drink by the customer of the current cuisine is “completed or has reached a predetermined stage near the end”. Alternatively, based on the image from the camera 5, the analysis unit 14 may calculate “the tableware area or volume (and color) and the dish area or volume (and color)” for the current dish or beverage on the dining table. In order to determine the "actual ratio", the area or volume (and color) of the current dish or beverage is less than a predetermined ratio with respect to the area or volume (and color) of the entire dish (for example, occupying only 10% or less). When it is determined that the food (e.g., hors d'oeuvres) is almost left uneaten, that is, the eating and drinking by the customer of the food is almost finished (a predetermined stage near the end is reached). You may make it.

(B) Next, in the first embodiment, a second method for the determination analysis unit 14 to determine the end of eating and drinking by the customer of each dish or beverage will be described with reference to FIG. In the second method, the determination analysis unit 14 uses the contents recorded in the table-specific order details recording unit 9 and the dish database 7 to prepare dishes (for example, hors d'oeuvres) arranged on the table 2. ), A “standard image before eating and drinking (reference image)” (see FIG. 3 (b)) is called, and the called standard image and the camera 5 are almost in real time (or at every predetermined time). The transmitted image of the actual dish (for example, hors d'oeuvres) 3 is compared with the image (see FIG. 3A) to extract difference information between the two images, and the difference (difference) is determined by a predetermined reference. If the amount is larger than the amount, it is determined that "the eating and drinking of the above-mentioned dish (for example, hors d'oeuvre) is almost completed (a predetermined stage close to the end is reached)".

(C) Next, in the first embodiment, a third method for the determination analysis unit 14 to determine the end of eating and drinking by the customer of each dish or beverage will be described with reference to FIG. In the first embodiment, as described above, the camera 5 “transmits only infrared rays at a predetermined timing by the control unit 6 or a microcomputer (not shown) (a microcomputer built in or external to the camera 5 directly). By attaching the "filter to make it", it operates not only as a visible light camera but also as an infrared camera (a kind of infrared detecting means) for detecting infrared rays generated from the food or beverage.

In the method shown in FIG. 4, the determination analysis unit 14 uses the infrared detection result (in this case, an infrared image picked up by infrared light) from the camera 5 operating as the infrared detection means (infrared camera) to detect the table 2. Estimating the temperature of the current dish above, based on this, measuring and judging the degree of progress of the customer's eating and drinking for the current dish on the dining table, and determining whether the set dish or the course dish The end of eating and drinking by a customer of a certain dish (or reaching a predetermined stage near the end) is determined. For example, according to the information from the visible light camera, even when it is determined that “the most amount of the dish or beverage is not eaten (there is a large amount of leftover food and drink)”, according to the infrared information, When the food or beverage (a food or beverage that is originally a warm food and cannot be eaten deliciously if it is not warm) drops below a predetermined temperature (for example, drops to room temperature) and becomes cold. " Or, when a state in which there is no change in the infrared rays (or the shape of the food or beverage) from the beverage has passed for a predetermined time (for example, several minutes) or more , the customer has only left the food or beverage left (drinking). It is determined that eating and drinking have already been completed. The information of the infrared detection result may be used alone to determine the timing of the end determination, or the video information from the camera 5 (a visible light camera when the filter is not attached) may be used. May be used in combination with to determine the timing of the end determination.

FIG. 4 shows an example of information from the camera 5 (image information by visible light and heat ray image by infrared light) as a state when displayed on the screen of the display 10. The screen of the display 10 is composed of divided screens 10a and 10b divided into upper and lower parts. Now, as shown in the split screen 10a of the display 10 in FIG. 4, a mug 34 containing a tableware 31, a dish (steak) 32 and a drink (soup) 33 on the table 2 of the customer 1 is present. I do. Then, from the camera 5 as the visible light camera, video information indicating the situation on the dining table 2 is transmitted to the control unit 6, and “tableware 31, dishes (steak) 32, drinks” as shown in FIG. (Soup) 33 and mug 34 ”are displayed on split screen 10a of display 10.

Immediately after imaging by the camera 5 as the visible light camera (for example, after 1-2 seconds), a control signal from the control unit 6 (or from a microcomputer directly built in or external to the camera 5). The camera 5, which is configured to operate as an infrared camera by attaching the “filter that transmits only infrared light” to the camera 5, acquires an infrared image of the food 32 or the beverage 33. The infrared image is transmitted to the control unit 6. The infrared image transmitted to the control unit 6 is displayed as shown in the lower split screen 10b of the display 10 in FIG. 4 (in the lower split screen 10b in FIG. Tableware 31 and mug 34 are not clearly shown because infrared rays are hardly emitted.) In the lower divided screen 10b, the temperature distribution of the food (steak) 32 and the beverage (soup) 33 is displayed in different colors (for example, a portion with a higher temperature is darker red, and a portion with a lower temperature is darker). (Shown in green).

The determination analysis unit 14 can determine whether or not a customer has finished eating and drinking a certain dish or beverage from the infrared information.

That is, for example, from the infrared detection result, the determination analysis unit 14 determines that the current temperature of the dish (steak) 32 on the dining table 2 is equal to a predetermined reference temperature (when the customer orders the dish of the dish ordered by the customer). At the standard temperature, the temperature decreased by more than the predetermined temperature (more than a predetermined temperature) compared to the temperature recorded in the cooking database 7 (for example, the current actual temperature of the cooking (steak) 32 is the same as the room temperature). 15 ° C., the standard temperature at the time of serving the dish (steak) 32 to the customer is 70 ° C., and the current temperature is lower than the standard temperature 70 ° C. by a predetermined temperature of 50 ° C. or more), and When there is no change in the infrared data from the dish (steak) 32 for a predetermined time or more (or the shape of the dish (steak) 32 obtained from the image from the camera 5 does not change). When condition has passed a predetermined time or more), the "food almost finished for the current cooking 32 'decision determines (such as, by performing in combination the visible light image to the infrared detection result Can be used, but can also be performed without using them together).

As described above, if only the video information from the visible light camera (the split screen 10a in FIG. 4) is used, most of the food (steak) 32 still remains. It's in progress. " However, when the infrared image information from the infrared camera is also included in the determination material as in the first embodiment, the control unit 6 reduces the temperature of the steak 32 almost to a room temperature or a temperature close to the tableware 41. The customer has guessed / determined that this steak 32 has no further intention to eat (will intend to leave it), and that eating and drinking by the customer of the steak 32 is almost finished. become. As described above, the determination analysis unit 14 provides information that cannot be understood from the visible light image alone (even though most of the steaks 32 still remain, the temperature is low, so the customer is trying to eat away. Is accurately estimated by also referring to the infrared image described above, so that it is accurately determined that "the eating and drinking of the customer of the current cuisine (including the beverage) is almost finished". I have.

Further, in the first embodiment, also for the soup 33 (see FIG. 4), the determination analysis unit 14 measures a current temperature of the soup 33 from a detection result of infrared rays from the soup 33, and based on the measured temperature, The current temperature of the soup 33 is compared with a comparison target temperature (for example, the current room temperature detected by a room temperature sensor), and the current temperature of the soup 33 is substantially the same as the comparison target temperature (for example, the current room temperature). When it has cooled down, and when there has been no change in the infrared data from the soup 33 for a predetermined time or longer (or there is no change in the shape of the soup 33 obtained from the image from the camera 5). when the state has passed a predetermined time or more), the "(even if already a significant amount of soup 33 inside the mug 34 remain) customer has ended already food soup 33 So that it is determined that the. That is, even if the determination analysis unit 14 recognizes that “most of the soup 33 still remains in the mug 34” based on the video information from the visible light camera 5, When it is also recognized that "the temperature of the soup 33 has cooled to a temperature close to room temperature", it is determined based on the infrared detection result that "the customer has already finished eating and drinking the soup 33". I have to.

In the above-described operation, the analysis unit 14 determines whether the visible light image from the camera 5 (information shown on the divided screen 10a in FIG. 4) and the infrared image (infrared detection result, information shown on the divided screen 10b in FIG. 4). Although the operation in the case of determining the timing based on both of the information has been described, in the present invention, for example, based only on the infrared image (infrared detection result) from the infrared detecting means (infrared camera), for example, The progress of eating and drinking can be inferred, for example, as "the temperature of the steak 32 (or the soup 33) is cold, so the customer may be trying to eat away".

(D) Next, in the first embodiment, a fourth method for determining the end of eating and drinking by a customer of each dish or beverage constituting the set-type dish or the course-type dish will be described with reference to FIG. explain. FIG. 5 shows an example of image information obtained by imaging the tableware such as a plate, a bowl, a cup, a knife, a fork, a spoon, and chopsticks on the table 2 by the camera 5, and the image information is displayed on a display 10 of the kitchen. This shows the state when displayed. The determination analysis unit 14 receives image information of tableware from the camera 5 as shown in FIG. 5 via the control unit 6 and, based on this, (1) plate, bowl, cup, knife, fork, Shape of each tableware such as spoon and chopsticks, (2) positional relationship (position and orientation) of each tableware to the entire table 2 (or to the customer), and (3) mutual positional relationship (position and direction) of each tableware And (4) tableware data indicating the mutual positional relationship (position and direction) between the plate and the knife, fork, and spoon is extracted by a known image processing / image recognition program. Then, the determination analysis unit 14 visually recognizes and estimates the progress of eating and drinking of the customer 1 with respect to the dish or beverage based on the tableware data as in (1), (2), (3), and (4), It is determined that eating and drinking by the customer of the dish or the beverage has been started and ended.

That is, in the first embodiment , based on the image of each tableware such as a plate, a bowl, a knife, a fork, a spoon, and chopsticks from the camera 5, the determination analysis unit 14 cuts out each of the tableware from the image. When recognizing and recognizing that a change has occurred in the position or orientation of each of those dishes, it is determined that eating and drinking by the customer has started for the dishes or beverages related to the dishes .

Further, in the first embodiment, the determination analysis unit 14 estimates the progress of eating and drinking of the customer based on the image of each tableware such as a plate, a bowl, a knife, a fork, a spoon, and chopsticks from the camera 5. The end of eating and drinking by the customer of the dish or the beverage is determined. That is, the analysis unit 14 determines the position of each tableware such as a plate, a bowl, a knife, a fork, a spoon, and a chopstick based on an image of each tableware such as a plate, a bowl, a knife, a fork, a spoon, and chopsticks from the camera. And the orientation and shape are image-recognized, and the "contents of image recognition" (tableware data such as (1), (2), (3), and (4) above) and the "standard for each dish extracted from the dish database 7" Information on the position or orientation of each tableware such as a plate, bowl, knife, fork, spoon, chopsticks (standard tableware data) "(the mutual positional relationship between each tableware and the table or customer, the mutual relation between each tableware and the dish) , And the standard data indicating the positional relationship between the tableware, etc.) are compared and collated to determine and infer the progress of eating and drinking of each dish or beverage of each customer. Then, the determination analysis unit 14 determines, based on the result of the determination / estimation, that the customer of the dish or the beverage has finished eating and drinking.

For example, (1) the determination analysis unit 14 determines the shape of each tableware such as the plate, bowl, cup, knife, fork, spoon, and chopsticks and the actual positional relationship of each tableware with respect to the entire table 2 (or customer 1). For example, based on the (position and orientation), when a plate is left on the table 2 at the center of the table 2 (a position relatively far away from the customer 1), for example, the message “the plate was served The customer has finished eating and drinking. " In addition, (2) the determination analysis unit 14 determines, for example, based on the mutual positional relationship (position and orientation) of the tableware, when two dishes are stacked on the table 2, The customer has already finished eating and drinking the dishes served on the two stacked dishes. " In addition, (3) the determination analysis unit 14 determines, based on the mutual positional relationship (position and orientation) among the plate, knife, fork, and spoon, for example, as shown in FIG. When the knife 43 and the fork 44 are placed on the plate 41 on which the dish (hamburger) 42 is placed in a direction intersecting with each other, "the plate 41 is placed on the plate 41 based on the manner or rule information of the course dish. The customer is still eating and drinking about the dish 32 that has been served. " On the other hand, based on the mutual positional relationship (position and direction) between the plate, knife, fork, and spoon, the determination analysis unit 14 may, for example, as shown in FIG. When the knife 43 and the fork 44 are placed side by side on the plate 41 on which the plate 42 is placed, the "dish 32 placed on the plate 41" is displayed based on the manner or rule information of the course dishes. As for, the customer has already finished eating and drinking. "

That is, for example, various manners and rules are defined for each country / region and each religion regarding “the mutual positional relationship of the plate, knife, fork, and spoon”. For example, in French cuisine, as shown in FIG. 5 (a), if a knife and a fork are placed on a tableware on which steaks and the like are intersected with each other (or a curved portion at the tip of the fork is placed upward). The waiter must not lower the dish (if brought to the kitchen) because it is still eating and drinking, while the knife and fork are arranged in parallel on the tableware as shown in Fig. 5 (b). If there is a manner or rule that the waiter may lower the dish (or if the curved portion of the tip of the fork is placed downward), the eating and drinking of the dish is finished. In the first embodiment, such various manners and rules are recorded in the cooking database 7. Therefore, the judgment analysis unit 14 extracts various manners and rule information from the cooking database 7 and converts them into data such as the mutual positional relationship between the plate 41, the knife 43, and the fork 44 on the dining table 2. On the basis of this, the progress of eating and drinking for the dish or beverage of each customer is determined, and the end of eating and drinking by the customer of the dish or beverage is determined based on this determination.

Note that there are various methods for recognizing the position and orientation of each tableware by the determination analysis unit 14. For example, the determination analysis unit 14 determines the shape and size of each tableware by using image information from the camera 5. Based on the information, the image is recognized by a known image recognition device and image processing (image analysis) software, and the type of each tableware can be determined (identified). In addition, from the image which imaged each tableware, such as the above-mentioned plate, knife, fork, spoon, chopsticks, etc. with a camera, the shape and direction of each tableware using a known image recognition device and image processing (image analysis) software. A technique for recognizing is well known, for example, as disclosed in Japanese Patent Application Laid-Open No. 5-266358 (in particular, see FIG. 6 of this publication). As a technique for recognizing the shape and orientation of the tableware as described above, for example, a contour detection device (using an artificial retinal chip that detects the contour of each object from the luminance difference of the object in the information captured by the camera) Some are known).

Next, FIG. 6A shows an example of an image from the camera 5 in the first embodiment (in FIG. 6A, for convenience of illustration, a state when the image is displayed on the display 10 in the kitchen). Is shown). In a restaurant to which the first embodiment is applied, it is assumed that there are six dining tables A, B, C, D, E, and F. In the example of FIG. 6A, the visible light image and the infrared image obtained by imaging the total of six dining tables AF from above or obliquely upward are listed on the screen of the display 10 in the order of the dining tables AF. (In the example of FIG. 6A, the visible light image and the infrared image are not separately displayed on the divided screens 10a and 10b as shown in FIG. 4, but are superimposed on one screen. (In the example of the screen in FIG. 6A, there are no customers at the dining tables C, D, and F).

In this case, the determination analysis unit 14 determines, for example, the eating and drinking progress of the customer at the dining table 2 of A based on the information from the camera 5. That is, based on the image information as shown in FIG. 6A, the determination analysis unit 14 determines whether the state of the dish on the dining table A is, for example, the “remaining amount” of the steak 3 which is a part of the set-type dish. And the temperature of the steak 3 has dropped to the same temperature as the room temperature, the eating and drinking of the steak 3 on the dining table A is almost finished (even if there is leftovers). Can be determined.

Next, FIG. 6B shows another example of the image from the camera 5 in the first embodiment (in FIG. 6B, when the image is displayed on the display 10 in the kitchen for convenience of illustration). Is shown). In the example of FIG. 6B, based on the video information from the camera 5, only the video showing each tableware (plate, knife, fork, spoon, etc.) on the six dining tables AF is extracted. The images of the tableware (cut out) are displayed in a list on the screen of the display 10 in the order of the dining tables A to F (that is, in the example of FIG. 6B, the dishes other than the tableware). Or, the video of the beverage is not extracted, but is discarded and cut off). In this case, the determination analysis unit 14 determines the progress of the customer's eating and drinking for each dish or beverage on each dining table 2 based on the video information of each dish extracted from the information from the camera 5. That is, based on the image information of each tableware as shown in FIG. 6B, the determination analysis unit 14, for example, at the table A, puts the steak, which is a part of the set-type dish, on the plate 41 of the tableware. When the knife 43 and the fork 44 are placed side by side in parallel, and the state has continued for a predetermined time or more, the food on the dining table A is not served by the customer based on the general rules of eating and drinking. It is determined that it is almost finished.

In the first embodiment, the video from the camera 5 is displayed by the control unit 6 on the display 10 in the kitchen in almost real time. Therefore, the cook 13 in the kitchen can grasp the progress of each customer's meal almost in real time by looking at the image on the display 10 even when the table 2 on the customer floor cannot be directly looked over from the kitchen. be able to. Therefore, in the first embodiment, the eating and drinking by the customer 1 of a certain stage of the first type of course-type dishes or the beverage has been completed or has reached a predetermined stage near the end (in the stage that is almost finished). At this time, the cook 13 recognizes this from the display 10 and starts cooking or preparing for the next stage of cooking or beverage, or instructs the waiter to serve or prepare. It becomes possible. Further, in the first embodiment, the eating and drinking by the customer of the dishes or beverages constituting the first stage of the second type of course-type cuisine has been completed or has reached a predetermined stage near the end (to the stage which is almost finished). At this time, the cook 13 recognizes this from the display 10 and starts cooking or preparing for the second stage “beverage after meal or dessert and beverage”, or serving or serving the meal. It is possible to instruct the waiter to prepare.

Further, in the first embodiment, the determination analysis unit 14 terminates the eating and drinking of the current dish or beverage on each dining table 2 by the customer 1 (including the case where the eating and drinking are finished with leaving). Is determined, a signal indicating the result of the determination is transmitted from the determination analysis unit 14 to the control unit 6 (in FIG. Refer to the transmission line of the judgment timing signal). And the control part 6 which received the said determination result transmits the alarm signal which shows that to the display 10 in a kitchen, and displays the image of the food or drink on the table 2 displayed on the display 10 on the display 10. By flashing the video of the corresponding dish or beverage (the dish or beverage targeted for the end of eating and drinking by the customer), the cook 13 is notified that the eating and drinking by the customer has been completed for the dish or beverage. Notification (warning) is made. In addition, the control unit 6 that has received the determination result transmits an alarm signal indicating that to the speaker 11 in the kitchen, outputs alarm information including a predetermined voice or sound, and outputs The end of eating and drinking by the customer is notified (warned) to the cook 13. Therefore, the notification (warning) by the control unit 6 enables the cook 13 to more appropriately cope with the end of eating and drinking by the customer of the dish or beverage.

Next, FIG. 7 is for explaining the "scanning device as infrared detecting means" 51 used in the second embodiment of the present invention. The configuration of the second embodiment is obtained by replacing the “camera 5 as an infrared detector” of the first embodiment with a “scanning device 51 as an infrared detector” described later. This is almost the same as the first embodiment.

7, 50 is a dish placed on a plate on the dining table 2, 50a is a beverage put in a cup on the dining table 2, and 51 is a "scanning device as infrared detecting means provided on the ceiling of a restaurant or the like. ]. The scanning device 51 includes a microcomputer 52 including a CPU (central processing unit) and memories (ROM and RAM); a scanning mechanism (rotating mechanism) 53 controlled by a control signal from the microcomputer 52; An infrared sensor (light receiving element) 54 that is sequentially (sequentially) scanned so that the detection target area (light receiving range) covers the entire table 2 by the mechanism 53. That is, the infrared sensor 54 is sequentially (sequentially) scanned by the scanning mechanism 53 so that the detection target area (light receiving range) covers the entire table 2.

The light receiving data from the infrared sensor 54 (infrared detection data) and the position / angle data from the scanning mechanism 53 (position / angle data for each scan when the infrared sensor 54 sequentially scans the table 2) Are input to the microcomputer 52 in synchronization with each other, and are recorded in the memory of the microcomputer 52 as position / angle data of the heat source. The position / angle data of the heat source is sent from the microcomputer 52 to the determination analysis unit 14 of FIG. 1 via the control unit 6 of FIG. Based on the position / angle data (the received light data and the position / angle data) of the heat source sent from the microcomputer 52, the determination analysis unit 14 (using a known analysis program or the like) The position of each dish 50 or the drink 50a on the dining table 2 and its temperature (temperature distribution on the surface or inside of each dish) are recognized.

Therefore, even when the "scanning device 51" shown in FIG. 7 is used as the "infrared ray detecting means" of the present invention, the food 50 or the drink Infrared rays from 50a (heat source object) can be detected. Then, the infrared detection result obtained by the scanning device 51 is sent to the determination analysis unit 14 via the control unit 6. Therefore, based on the infrared detection result obtained by the scanning device 51, the determination analysis unit 14 calculates the progress of the customer's eating and drinking for the food 50 or the beverage 50a from the current temperature distribution of the food 50 or the beverage 50a Is determined, and based on this determination, it can be determined whether or not the customer's eating and drinking of the current dish or beverage has ended.

Next, FIG. 8 shows a "scanning device as a tableware direction detecting means (three-dimensional data acquisition means) for detecting the direction, position, shape, etc. of each tableware" 61 used in the third embodiment of the present invention. It is for explanation. The configuration of the third embodiment is obtained by replacing the “camera 5 as the optical information detecting unit” of the first embodiment with a “scanning device 61 as a tableware direction detecting unit” to be described later. Is substantially the same as in the first embodiment.

In FIG. 8, reference numeral 60 denotes a plate on the dining table 2, 60a denotes a cup on the dining table 2, and 61 denotes a "scanning device as a table direction detecting means" provided on the ceiling of a restaurant or the like. The scanning device 61 includes a microcomputer 62 including a CPU, a memory, and the like, and a spot light source 63 controlled by the microcomputer 62 to irradiate a spot light (such as a laser beam) to any point on the dining table 2. (In place of the spot light source 63, a device that emits a directional radio wave (pulse wave) such as a microwave or a millimeter wave that is emitted using a directional antenna may be used. A scanning mechanism for sequentially (sequentially) scanning the table 2 with the spot light so that the spot light emitted from the spot light source 63 covers substantially the entire upper surface of the table 2 while being controlled by the microcomputer 62. A rotating mechanism) 64 and a reflected light of the spot light from the spot light source 63 (the object on the dining table 2). An optical sensor (light receiving element) 65 for receiving the reflected light) from such 0,60A, and a. That is, the spot light from the spot light source 63 is sequentially scanned by the scanning mechanism 64 so as to sequentially illuminate almost the entire table 2.

Each light receiving data from the optical sensor 65 and position / angle data at each scanning time from the scanning mechanism 64 (position / angle data at each scanning time when the spot light from the spot light source 63 is sequentially scanned) ) Are input to the microcomputer 62 in synchronization with each other, and are recorded in the memory of the microcomputer 62. The scan data (the received light data and the position / angle data) recorded in this memory is sent from the microcomputer 62 to the determination analysis unit 14 in FIG. 1 via the control unit 6 in FIG. In the analysis unit 14, based on the received light data and the position / angle data sent from the microcomputer 61 via the control unit 6 (using a known analysis program or the like), the tableware is used. The direction, the position, the shape, the size, the mutual positional relationship of the (plate 60 and the cup 60a) on the table 2 are recognized.

As described above, in the third embodiment, the spot light (or the “directional radio wave”) may be scanned over the entire table and the reflected light (reflected wave) may be received (received). Thereby, the determination analysis unit 14 (the part that executes the analysis program for the received light data and the position / angle data) can recognize the orientation, position, shape, size, mutual positional relationship, and the like of each tableware on the table 2. (Note that the above-described scanning device 61 may be regarded as a kind of radar (laser radar or the like)). Therefore, even when the "scanning device 61" shown in FIG. 8 is used as the "tableware direction detecting means" of the present invention, the "camera 5" of FIG. 1 is used as the "tableware direction detecting means" of the present invention. Similarly, the direction, position, shape, size, and mutual positional relationship of each tableware (including knives, forks, spoons, chopsticks, etc.) such as the plate 60 or the cup 60a on the table can be recognized.

As described above, in the third embodiment, the determination analysis unit 14 uses the scan data (the received light data and the position / angle data) transmitted from the scanning device 61 to detect the tableware such as the dish 60 or the cup 60a. Of each tableware, such as the direction, position, shape, size, and mutual positional relationship of the tableware (data such as the direction, position, shape, size, etc. of each tableware obtained by the scanning device 61). Based on the orientation and position information of each dish, the customer's eating and drinking progress for the dish placed in the dish 60 or the cup 60a is determined. Whether eating and drinking by the user is almost finished "is determined.

That is, in the third embodiment, the determination analysis unit 14 recognizes the mutual positional relationship between the knife and the fork (the knife and the fork are not shown in FIG. 8) based on the scan data. If they are placed side by side on a plate in parallel with each other, it is determined that the eating and drinking by the customer for the dish or beverage has been completed based on the manners and rule information of the course dish. Further, based on the scan data, if the position of the plate 60 in FIG. 8 is located at the center of the dining table 2 away from the customer, the determination analysis unit 14 It is determined that the eating and drinking by has been completed.

In the third embodiment, the direction and position of each tableware such as the direction, position, shape, size, and mutual positional relationship of the tableware such as the dish 60 or the cup 60a are recognized based on the data from the scanning device 61. Instead of (or together with, obtaining the orientation / position information of each tableware), the "shape of each dish or beverage" actually present in the plate 60 or the cup 60a is recognized. (Shape data may be obtained). Then, based on the acquired shape data of the dish or beverage, the progress of eating and drinking of the customer is determined. You may make it.

As a related technique of the third embodiment, that is, a technique for irradiating spot light and acquiring three-dimensional shape data of a target object based on the reflected light (the above-described tertiary shape of each tableware on the table) As a technique for detecting the original shape, orientation, and the like, for example, “Technical Research Report of IEICE, IEICE, Vol. 101, No. 101, issued by the Institute of Electronics, Information and Communication Engineers on December 13, 2001. 524 ", pp. 101-107," Development of high-speed, high-density head shape measurement range finder system (author: Atsushi Marugame) "exists. This technology irradiates a sine-wave pattern of light (or laser light) from multiple projectors while scanning (shifting) it toward a surrounding object (for example, tableware on a table), and irradiates the pattern image. Is captured by a plurality of cameras, phase values are calculated, three-dimensional coordinates are calculated based on the principle of triangulation, and three-dimensional shape data of surrounding objects are obtained (in the above-mentioned paper, “multi-view sine Wave grating phase shift method "). That is, an object (for example, each tableware on a table) is irradiated while shifting the shading pattern of a sine wave three or more times by a fixed amount, an image of the pattern is taken, a phase value is calculated, and three-dimensionally calculated based on the principle of triangulation. The coordinates are calculated. The sine wave pattern shift (scan) mechanism may be, for example, a sine wave pattern irradiating device that stands on the edge of the cam, the cam rotates with a stepping motor, and the edge of the cam has a spiral staircase. It is possible to adopt a machine that is machined so that the upper sine wave pattern shifts by a constant amount when the cam rotates. If the stepping motor is rotated at a high speed by this shift mechanism, the sine wave pattern can be shifted (scanned) at a high speed. Acquisition of a sine wave pattern image is performed by capturing an image with a camera when the pattern is at each stage. A control unit such as a microcomputer performs the above-described sine wave pattern shift (scan; cam rotation control), camera imaging control, and projector lighting control (the above description is based on the above description. Quoted the text of the dissertation).

Next, a fourth embodiment of the present invention will be described with reference to FIG. The configuration of the fourth embodiment is basically the same as the configuration of the first embodiment described with reference to FIG. FIG. 9 shows an example of video information obtained by imaging the table 2 by the camera 5 of FIG.

The video information in FIG. 9 shows a state in which the customer places his favorite dishes or drinks on his own dish and places it on his own dining table 2 when the restaurant provides buffet dishes. Is shown.

映像 Video information from the camera 5 as shown in FIG. 9 is transmitted to the determination analysis unit 14 via the control unit 6. Based on the video information (the video shown in FIG. 9) from the camera 5, the determination analysis unit 14 uses known image processing / image recognition software to place the dishes 71, 72, 73 on the dining table 2. And 74 and a cup 75, two breads on a plate 71, one orange and one bunch of grapes on a plate 72, a plate 73 That there are two shrimp fries on the plate, one hamburger and three meatballs on the plate 74, and one cup of soup in the mug 75 I do.

In general, since various kinds of dishes or beverages provided as buffet dishes are limited in number to several dozens, conventionally known image processing and image recognition methods (for example, the image of the image from the camera 5). Extract only the food or beverage part from the inside, compare each extracted with standard images of various foods or beverages registered in advance, and match those with high similarity to the corresponding type It is possible to specify the type and number of a plurality of dishes or beverages included in the video from the camera 5 by the method of determining that

Note that image processing and image recognition techniques for determining and identifying what kind of food or food the food or food in the image from the image of the food or food captured by the camera include, for example, sushi at a conveyor belt sushi restaurant. Discrimination / identification of the type is introduced in JP-A-2000-107006 and JP-A-2000-245609. Further, a technology for automatically recognizing the shape and dimensions of the dishes in the image processing and image recognition processing of the image taken by the camera is disclosed in Japanese Patent Application Laid-Open No. Hei 5-266358.

On the other hand, attribute information of the customer sitting at the dining table 2 is input to the determination analysis unit 14 from the customer attribute generation unit 16 (see FIG. 1). As the attribute information of the customer, when the waiter guides the customer of the buffet dish to a predetermined dining table 2, the waiter inputs the gender and age group of the customer from the portable information terminal, and identifies it by identifying the dining table 2. The information is transmitted to the customer attribute generating unit 16 wirelessly in association with information (this is also used as the identification information of the customer). Further, the determination analysis unit 14 is constantly input with the date and time data from the date and time data generation unit 15 (see FIG. 1).

As described above, the determination analysis unit 14 determines that the customer (customer specified by the identification number of the dining table 2) can select his / her favorite type of dish or beverage from various dishes or beverages provided as buffet dishes. When a desired number of beverages are separated into their own dishes and placed on their own dining table 2, the types and numbers of the plurality of dishes or beverages are recognized by the image recognition processing as described above, and the recognition is performed. The type and number of cooked dishes or beverages (eg, two breads, one orange, one bunch of grapes, two fried shrimp, one hamburger, three meatballs, and one soup as described above) , And is sent to the database management unit 20 in association with the attribute and date of the customer.

Then, the database management unit 20 further associates the information sent from the determination analysis unit 14 with the weather data (including the temperature data) and the store location attribute data of the area where the store is located in the time zone, The information is recorded in the marketing database 17.

As described above, in the fourth embodiment, when the restaurant provides the buffet food, the customer prefers his or her favorite type of food or drink from the various foods or drinks provided as the buffet food. Of the dishes or beverages that each customer has sorted out, image-recognizing the type and number of dishes or beverages that the customer has sorted out. Can be recorded in association with the attributes of each customer, date and time, day of the week, season, weather, store location attributes, and the like. Therefore, the store may ask what types of foods or beverages many customers prefer (i.e., what types of foods or beverages contribute more to sales and profits), and what What kind of food or beverage the customer with the attribute prefers, and what kind of food or beverage is more preferred in what season, time of day, day of the week, weather, and store location characteristics Can be grasped with high accuracy.

In the fourth embodiment, the image information from the camera 5 is used even when a scanning device 61 as a three-dimensional data acquisition device shown in FIG. 8 is used instead of the camera 5 shown in FIG. As in the case, based on the scan data from the scanning device 61, it is possible for each customer to recognize the type and number of each dish or beverage placed on the dining table 2 according to his / her preference.

(Expansion) Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications are possible. For example, in each of the above embodiments, image data of dishes, beverages, or dishes acquired by the visible light camera 5, infrared detection of dishes or beverages acquired by the infrared camera 5 or the scanning device 53 (see FIG. 7). Result data, dish or beverage shape data obtained by the camera 5 or the scanning device 61 (see FIG. 8) (more precisely, the determination analysis unit 14 extracts “cook or beverage shape data” Basic data), shape / position / orientation data of each tableware acquired by the camera 5 or the scanning device 61 (see FIG. 8) (more precisely, the determination analysis unit 14 determines “position / orientation data of each tableware”). Is first transmitted to the control unit 6 on the kitchen side, and the basic data is further transmitted from the control unit 6 to the analysis unit 14. , The respective data, without using the control unit 6 directly, may be sent to the judging analyzing unit 14.

Further, in the present invention, the determination analysis unit 14 controls a remote computer (for example, a number of restaurants) connected to the camera 5 (the scanning device 53 or the scanning device 61) via, for example, the Internet. (A server at the headquarters that develops a chain).

In the first embodiment, the determination analysis unit 14 uses the current date and time data (current date and time) from the date and time data generation unit 15 to construct a set-type or course-type dish. Although the time required by the customer to eat and drink is measured for each dish or beverage to be served, the present invention is not limited to this. For example, the shop side can provide the customer with a set-type dish, the first or second type. Using the time data (functions such as stopwatch) measured and generated from the point in time when the food or beverage at each stage of the course-type dishes is provided as zero (initial value) Alternatively, the time required for a customer of each dish or beverage constituting the course-type dish to eat and drink may be measured.

(Prior art information to be referred to for the invention of the present application) Finally, prior art information to be referred for understanding the present invention is described below.

1. Japanese Patent Application Laid-Open No. 2000-107006: The system disclosed in this publication uses a rotary conveyor based on video information from a camera on a conveyor in a restaurant of a type in which sushi placed on a plate is put on a rotary conveyor and transported in a circuit. The type of each sushi put into the sushi, the identification of each sushi, and the calculation of the round transport time of each sushi on a rotating conveyor are performed. In this system, for example, in order to identify the type of sushi (octopus, squid, and tuna), standard optical characteristic information (measurement data) at a plurality of observation sites for each category of octopus, squid, and tuna ( RGB values, YUV values) are recorded in advance. Then, the intersections of radiation at 45-degree intervals from the center of the dish and concentric circles shared about the center of the dish are set as sampling points, and optical property information of each sushi at these sampling points is input from the camera. . Then, the optical property information of each sampling point is compared with representative optical property information of a plurality of categories recorded in advance, and the Mahalanobis distance and the probability pi (j) are calculated. Then, the probabilities pi (k) of all sampling points for each category are integrated to calculate an average probability p (k) for each category, and the category of the maximum probability is obtained, thereby achieving identification of the type of sushi. Like that. In addition, in order to specify each sushi, the history of the sushi that has identified the type is recorded, and each sushi newly introduced on the conveyor is specified based on the pattern of the history. I have.

2. "Face Recognition System" currently sold by Nikon System Co., Ltd .... Measures, for example, more than 80 places of each person's face, quantifies the features, and finds a person with high similarity from the face database System. In this system, a face is recognized from an image captured by a video camera and collated with data of a face database registered in advance. That is, the facial characteristics such as the width of the face, the thickness of the lips, the thickness of the eyebrows, and the angle of the corners of the eyes are extracted from the image and quantified, and a face database having a high degree of coincidence is selected. It can detect even if multiple people are shown on the camera at the same time.

3. Patent No. 3115776-A seat management POS system, in which when an employee accepts an order from a customer at a seat with an arbitrary seat number, a seat number and an order product are stored in a POS RAM for each customer order. Is registered, and at the time of registration, the internal timer of the POS is operated. The internal timer is a software timer, and based on a hardware timer built in the POS, a software counter area is provided on the RAM for each seat number, and the time from the last order is monitored for each seat. As a result of the time measurement by the software timer, when it is detected that a certain period of time has elapsed without reordering from the customer's seat, the order is determined to be the last order for this customer, and the customer leaves soon. Then, the image of the seat is displayed on the display unit of the POS terminal. Such a system is disclosed.

4. Japanese Patent No. 3231709: A food and beverage service system that sequentially provides a plurality of ordered menus such as course meals at a certain time interval, and provides information on the order of serving each menu and the expected eating time of a customer (standard eating time) Time) and the standard cooking time, a scheduled serving time for the customer is calculated, and based on the calculated scheduled serving time and the standard cooking time, a cooking start time for each order menu is calculated. A system for outputting a cooking instruction is disclosed.

FIG. 1 is a schematic block diagram for explaining a configuration according to a first embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the first embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the first embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the first embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the first embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the first embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the first embodiment of the present invention. FIG. 4 is a schematic block diagram illustrating a part of a configuration according to a second embodiment of the present invention. FIG. 9 is a schematic block diagram illustrating a part of a configuration according to a third embodiment of the present invention. FIG. 9 is a diagram for explaining a fourth embodiment of the present invention.

Explanation of reference numerals

1 customer 2 dining table 3 steak (cooking)
4 Tableware 5 Camera 6 Control unit 7 Cooking database 8 Order contents input unit 9 Order contents recording unit 10 Kitchen side displays 10a, 10b Split screen 11 Kitchen side speaker 12 Kitchen side printer 13 Cook 14 Judgment analysis unit 15 Date and time data generation unit 16 Customer attribute generation unit 17 Marketing database 18 Weather data generation unit 19 Store location characteristic generation unit 20 Database management units 21, 34, 75 Mug 31 Tableware 32 Steak, dish 33 Soup (drink)
41, 60, 71, 72, 73, 74 dishes (tableware)
43 Knife 44 Fork 50 Cooking 50a Beverage 51 Scanning device 53 Scanning mechanism 54 Infrared sensor 60a Cup 61 Scanning device 63 Spot light source 64 Scanning mechanism 65 Optical sensor

Claims (12)

When "a plurality of dishes and one or a plurality of beverages" are provided in a plurality of stages at different times, a first type of course-type dish is provided to a customer in a restaurant. Data analysis means for automatically obtaining information of "time required for eating and drinking by the customer" for each of the dishes or beverages at each stage constituting the course-type dish,
Based on the output from the data analysis means, for each dish or beverage that constitutes the course-type dish, for each customer, a marketing database for recording "time required for the customer to eat and drink",
A marketing data collection system for a restaurant, comprising: "A plurality of dishes or a plurality of dishes and one or more beverages are provided as a first step, and a" beverage or dessert and a beverage "are provided as a second step. When the dishes or beverages of the first stage or the second stage constituting the “course-type dishes of type” are provided to a customer in a restaurant, “all or one of the first stages or the second stages is The time required for each customer to eat and drink for each of the dishes or beverages constituting the part, and the eating and drinking by the customer for each of the dishes or beverages in the group of the dishes or beverages constituting all or a part of the first stage starts. Or the order in which eating and drinking by the customer with respect to each dish or beverage in the group of dishes or beverages constituting all or a part of the first stage has ended or has reached a predetermined stage. ” And data analysis means for determining automatically,
On the basis of the output from the data analysis means, for each dish constituting the course-type dish, for each customer, "the time required for the customer to eat and drink, the dish constituting all or a part of the first stage" Or the order in which eating and drinking by the customer for each dish or beverage in the group of beverages was started, or for each dish or beverage in the group of dishes or beverages constituting all or part of said first stage. Marketing database for recording the order in which the eating and drinking by the customer has been completed or has reached a predetermined stage near the completion of eating and drinking,
A marketing data collection system for a restaurant, comprising: When "set dishes" in which "a plurality of dishes and one or more beverages" are provided almost simultaneously, are provided to a customer in a restaurant, "each of which constitutes all or a part of the set dishes" The time required by the customer to eat and drink each of the dishes or beverages, the order in which the eating and drinking of the customers for each dish or beverage in the group of dishes or beverages constituting all or a part of the set-type dish was started, or The information of the order in which the eating and drinking by the customer with respect to each dish or beverage in the group of dishes or beverages constituting the whole or a part of the set-type dish has been completed or has reached a predetermined stage near the end is automatically performed. Data analysis means for obtaining
Based on the output from the data analysis means, for each of the dishes or beverages that make up the set-type dish, for each customer, `` For each dish or beverage that makes up all or part of the set-type dish, The time required for eating and drinking, the order in which the eating and drinking by the customer regarding each dish or beverage in the group of dishes or beverages constituting all or a part of the set dish, or the entire set dish or A marketing database for recording an order in which a customer's eating and drinking for each dish or beverage in a group of dishes or beverages forming a part has reached or reached a predetermined stage near the end.
A marketing data collection system for a restaurant, comprising: In claim 1, 2, or 3,
The data analysis means, for each dish or beverage provided on the dining table on the dining floor of the restaurant, `` whether the eating and drinking by the customer has been completed or whether the eating and drinking by the customer has reached a predetermined stage near the end And a terminating means for automatically judging "whether". A marketing data collection system for a restaurant. In claim 2 or 3,
The data analysis unit is a unit that determines whether each dish or beverage is provided to the customer or whether the customer has started eating or drinking, for each dish or beverage provided on the dining table on the dining floor of the restaurant. A means for determining whether or not the customer has finished eating or has reached a predetermined stage near the end of eating and drinking. In claim 4,
Including image acquisition means for acquiring an image of each dish or beverage on the dining table,
The end determination unit is based on the image information from the image acquisition unit, when each dish or beverage on the table is almost gone or when the state of each dish or beverage on the table does not fluctuate for a predetermined time or more, A marketing data collection system for a restaurant, wherein it is determined that the food or drink of the customer has reached or reached a predetermined stage near the end of eating and drinking by the customer. In claim 4,
Infrared data acquisition means for acquiring infrared data emitted from each dish or beverage on the table,
The end determination means, based on the infrared data from the infrared data acquisition means, when the emission of infrared light from each dish or beverage on the table has almost disappeared, the temperature of each dish or beverage on the table is predetermined. When a predetermined time or more has elapsed since the temperature has dropped below the temperature, or when the temperature of each dish or beverage on the dining table has passed a predetermined time or more after the temperature has decreased by a predetermined temperature or more from the temperature at the time of serving, the dish or beverage Is a system for determining that the eating and drinking by the customer has reached the end or a predetermined stage near the end. In claim 4,
Including shape data acquisition means for acquiring shape data of each dish or beverage on the dining table,
Based on the shape data obtained by the shape data obtaining means, the end determination means, when each dish or beverage on the table is almost gone or the shape of each food or drink on the table fluctuates for a predetermined time or more If not, the marketing data collection system for a restaurant is determined to determine that the food or drink by the customer has reached or reached a predetermined stage near the end of the food or drink. In claim 4,
For obtaining tableware data indicating "shape, position, and / or orientation of each tableware such as plate, cup, bowl, fork, knife, spoon, chopsticks, etc." provided with each dish or beverage on the table. Including tableware data acquisition means,
The end determination means is based on the tableware data obtained by the tableware data obtaining means, on the table, the `` dishes, cups, bowls, forks, knives, spoons, shape and position of each tableware such as chopsticks, And / or the orientation "does not fluctuate for a predetermined time or more, or when the knife and fork are placed side by side substantially parallel to each other for a predetermined time or more, the food or drink of the food or beverage by the customer is not consumed. A marketing data collection system for a restaurant, wherein it is determined that the process has ended or a predetermined stage near the end has been reached. In claim 4,
Including tableware data acquisition means for acquiring tableware data indicating `` each position and orientation of forks and knives '' provided with each dish or beverage on the table,
The end determination means, based on the tableware data acquired by the tableware data acquisition means, when the `` state in which the fork and knife are arranged substantially parallel to each other '' continues for a predetermined time or more on the table, A marketing data collection system for a restaurant, wherein it is determined that the food or drink of the customer has reached or reached a predetermined stage near the end of eating and drinking by the customer. In any one of claims 1 to 10,
The marketing database, the information obtained by the data analysis means, the "attribute of each customer, the date and time of providing the eating and drinking service, the day of the week when providing the eating and drinking service, the time or time zone when providing the eating and drinking service, The season at the time of providing the eating and drinking service, the weather at the time of providing the eating and drinking service, the temperature at the time of providing the eating and drinking service, or the location characteristics of a store that provides the eating and drinking service, are recorded in association with each other. Marketing data collection system for restaurants. "Buffet food, in which the customer places various dishes or drinks arranged in a predetermined place in the restaurant on his own table according to his preference and places it on his own table," Data analysis means for obtaining information of "the type and / or number of dishes or beverages placed on their own dining table by the customer's own dishes, when provided to customers in the store, On the basis of the information obtained by the data analysis means, for each buffet dish, for each customer, "type and / or number of dishes or drinks placed by customers on their own dining table in their own dishes And a marketing database for recording the marketing data collection system for a restaurant.

Images