JP2018077625A - Menu generation system, menu generation method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy of a designation of a menu determined from an appearance image obtained by photographing a dish composed of two items or more.SOLUTION: A menu generation system for generating a menu of a restaurant comprises: a storage part for, as for each of a plurality of dishes, storing a sample dish image obtained by imaging an appearance and a dish designation and dish attributes in association; an input processing part for, as for a predetermined menu composed of two or more dishes among a plurality of dishes, receiving an appearance image including an appearance of each dish; a specification part for, on the basis of the received appearance image and the stored sample dish image and dish designation, specifying a dish designation of each dish included in the received appearance image; and a calculation part for, as for the specified dish designation of each dish, calculating similarity between the dish attributes associated with the dish designation of each dish and the dish attributes associated with the dish designation of the other dish.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a menu generation system, a menu generation method, and a control program.

  Conventionally, referring to a database in which a sample image obtained by capturing a specific object in advance and the name of the specific object are stored in association with each other, the image recognition for outputting the name of the specific object included in the image captured by the user The system is known.

  For example, in Patent Document 1, a plurality of dishes constituting a set meal are determined based on the captured set meal image, and a menu item (name of the set meal) including the determined dish items is output. An order management system is described. In this order management system, a completed dish image is stored for each dish item, and a plurality of dish items constituting the menu item are stored for each menu item (set meal or the like). The order management system acquires an image of a menu item such as a set meal that has been imaged, determines a plurality of completed dish images included in the acquired image, and sets a cooking item of each completed dish image for each determined completed dish image. Identify. Then, when the menu item configured by each identified cooking item can be determined, the order management system outputs the determined menu item. Thereby, the person in charge of cooking automatically confirms whether or not the set meal to be served includes a plurality of cooking items constituting the set meal by imaging the set meal before serving the set meal. can do.

JP 2011-76323 A

  However, in the conventional system, when the name of the captured food appearance image cannot be specified by the previously stored food image, the user only has to be notified of information indicating that the name specifying process has failed. It was. Therefore, when the process for specifying the name of the dish image fails, the user has to visually check all the dishes.

  The present invention has been made to solve such a problem, and a menu generation system and menu that can improve the accuracy of menu names determined from appearance images obtained by photographing two or more dishes. It is possible to provide a generation method and a control program.

  The menu generation system according to the present invention is a menu generation system for generating a restaurant menu, and stores, for each of a plurality of dishes, a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute in association with each other. An input processing unit for receiving an appearance image including an appearance of each dish for a predetermined menu including two or more dishes of a plurality of dishes, a received appearance image, a stored sample dish image, and Based on the dish name, the identification unit that identifies the dish name of each dish included in the received appearance image, the dish attribute associated with the dish name of each dish, and other information about the dish name of each identified dish And a calculation unit that calculates the similarity with the dish attribute associated with the dish name of the dish.

  Further, in the menu generation system according to the present invention, the specifying unit applies the received appearance image to the received appearance image based on the similarity calculated by the calculating unit so that the dish attribute associated with the specified dish name of each dish does not overlap. It is preferable to respecify the dish name of each dish included.

  Further, in the menu generation system according to the present invention, the specifying unit calculates the degree of coincidence between the image of each dish included in the received appearance image and the sample dish image, and is associated with the specified dish name of each dish. When the dish attributes are duplicated, it is preferable to change the dish names other than the dish names having the highest degree of coincidence calculated for the corresponding sample dish images among the dish names of the dishes having the duplicate dish attributes.

  In the menu generation system according to the present invention, the dish attribute is preferably the order in which dishes are provided in a menu composed of two or more dishes.

  Moreover, the menu generation system according to the present invention preferably includes a generation unit that generates display information of a menu in which dish names of each identified dish are arranged according to a provision order associated with each dish.

  In the menu generation system according to the present invention, the dish attribute is preferably a dish type.

  In addition, the menu generation system according to the present invention preferably includes a transmission unit that notifies the user of notification information indicating that there is an error in specifying the dish name by the specifying unit when the similarity is greater than or equal to a predetermined level.

  Further, in the menu generation system according to the present invention, the notification information includes at least one dish name among dish names associated with dish attributes having a similarity degree equal to or higher than a predetermined degree among the identified dish names. It is preferable that the name suggestion information changed to the dish name associated with another dish attribute different from the dish attribute having a similarity degree equal to or higher than a predetermined level is included.

  In the menu generation system according to the present invention, the storage unit stores, for each of the plurality of dishes, the dish name written in the first language and the dish name written in the second language in association with each other, It is preferable to have a transmission unit that transmits the dish name written in the first language of the dish or the dish name written in the second language to the terminal.

  The menu generation system according to the present invention is a menu generation system for generating a restaurant menu, and stores, for each of a plurality of dishes, a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute in association with each other. An input processing unit for receiving an appearance image including an appearance of each dish for a predetermined menu including two or more dishes of a plurality of dishes, a received appearance image, a stored sample dish image, and Based on the dish name, the identifying unit that identifies the dish name of each dish included in the received appearance image, the dish attribute associated with the dish name of each dish for the identified dish name of each dish, and a predetermined And a calculation unit for calculating the degree of similarity with the cooking attribute.

  Further, in the menu generation system according to the present invention, the specifying unit associates all of the dish attributes associated with the specified dish names of each dish based on the similarity calculated by the calculating unit. It is preferable to respecify the dish name of each dish included in the received appearance image so that the dish attribute is obtained.

  In the menu generation system according to the present invention, it is preferable that the specifying unit sets the most cooking attributes among the cooking attributes associated with the specified dish names as predetermined cooking attributes.

  In the server according to the present invention, the dish attribute is preferably a dish genre.

  In the server according to the present invention, the storage unit stores a dish name written in the first language and a dish name written in the second language in association with each other for each of the plurality of dishes. It is preferable to have a transmission unit that transmits the dish name written in the first language or the dish name written in the second language to the terminal.

  In addition, the server according to the present invention preferably includes a transmission unit that notifies notification information indicating that there is an error in specifying the dish name by the specifying unit when the similarity is less than a predetermined level.

  In the server according to the present invention, the notification information is changed from a dish name that is not associated with a predetermined dish attribute among the dish names of each identified dish to a dish name associated with the predetermined dish attribute. It is preferable to include name suggestion information.

  A menu generation method according to the present invention is a menu generation method by a computer that has a storage unit and generates a restaurant menu, and includes a sample dish image, a dish name, and a dish that captures the appearance of each of a plurality of dishes. Attributes are associated with each other and stored in the storage unit, and for a predetermined menu including two or more dishes of a plurality of dishes, an appearance image including the appearance of each dish is received, and the received appearance image is stored. Based on the sample dish image and the dish name, the dish name of each dish included in the received appearance image is specified, and for the identified dish name of each dish, the dish attribute associated with the dish name of each dish and others Calculating similarity with the dish attribute associated with the dish name of the dish.

  A control program according to the present invention is a computer control program that has a storage unit and generates a restaurant menu, and includes a sample dish image and a dish name obtained by imaging the appearance of each of a plurality of dishes on the computer. The cooking attributes are associated with each other and stored in the storage unit, and an appearance image including the appearance of each dish is received for a predetermined menu including two or more dishes of a plurality of dishes, and the received appearance image is stored. And identifying the dish name of each dish included in the received appearance image based on the sample dish image and the dish name, and the dish attribute associated with the dish name of each dish for the identified dish name The similarity is calculated with the dish attribute associated with the dish name of another dish.

  A menu generation method according to the present invention is a menu generation method by a computer that has a storage unit and generates a restaurant menu, and includes a sample dish image, a dish name, and a dish that captures the appearance of each of a plurality of dishes. Attributes are associated with each other and stored in the storage unit, and for a predetermined menu including two or more dishes of a plurality of dishes, an appearance image including the appearance of each dish is received, and the received appearance image is stored. Based on the sample dish image and the dish name, the dish name of each dish included in the received appearance image is specified, and the dish attribute associated with the dish name of each dish for the identified dish name of each dish, Calculating similarity to a predetermined dish attribute.

  A control program according to the present invention is a computer control program that has a storage unit and generates a restaurant menu, and includes a sample dish image and a dish name obtained by imaging the appearance of each of a plurality of dishes on the computer. The cooking attributes are associated with each other and stored in the storage unit, and an appearance image including the appearance of each dish is received for a predetermined menu including two or more dishes of a plurality of dishes, and the received appearance image is stored. And identifying the dish name of each dish included in the received appearance image based on the sample dish image and the dish name, and the dish attribute associated with the dish name of each dish for the identified dish name , Calculating the degree of similarity with a predetermined dish attribute.

  With the menu generation system, the menu generation method, and the control program according to the present invention, it is possible to improve the accuracy of menu names determined from appearance images obtained by photographing two or more dishes.

1 is a schematic diagram for explaining an overview of a menu generation system 1. FIG. It is a figure which shows an example of schematic structure of the menu production | generation system. 2 is a diagram illustrating an example of a schematic configuration of a terminal device 2. FIG. (A)-(b) is a figure which shows an example of the screen displayed on the terminal device 2. FIG. 6 is a diagram illustrating an example of a screen displayed on the terminal device 2. FIG. (A)-(b) is a figure which shows an example of the screen displayed on the terminal device 2. FIG. 6 is a diagram illustrating an example of a screen displayed on the terminal device 2. FIG. 6 is a diagram illustrating an example of a screen displayed on the terminal device 2. FIG. 6 is a diagram illustrating an example of a screen displayed on the terminal device 2. FIG. 2 is a diagram illustrating an example of a schematic configuration of a server 3. FIG. (A)-(b) is a figure which shows an example of the data structure of various tables. It is a figure which shows an example of the operation | movement sequence of the menu production | generation system. It is a figure which shows an example of the operation | movement flow of a specific process. It is a figure which shows an example of the operation | movement flow of a registration process. 1 is a diagram illustrating an example of a schematic configuration of a menu generation system 10. FIG.

  Hereinafter, various embodiments of the present invention will be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to these embodiments, and extends to the invention described in the claims and equivalents thereof.

(Outline of menu generation system)
The menu generation system receives various types of information transmitted from the terminal device 2 owned by a restaurant related person such as a restaurant manager, identifies the name of the menu based on the received various types of information, The name is transmitted to the terminal device 2. The terminal device 2 is, for example, a multi-function mobile phone called “smart phone” or a mobile phone called “feature phone”. Any communication device can be used. Hereinafter, a restaurant-related person who owns the terminal device 2 may be simply referred to as a user.

  FIG. 1 is a schematic diagram for explaining an outline of the menu generation system. Hereinafter, the search function of the menu generation system will be described with reference to FIG.

  First, in response to a user operation, the terminal device 2 captures the appearance of each dish for two or more dishes constituting a predetermined menu, and stores the captured appearance image. The predetermined menu is, for example, a course menu, a platter menu, or a set menu. Hereinafter, the predetermined menu may be simply referred to as a menu. Next, the terminal device 2 transmits the stored appearance image to the server 3.

  For each of the plurality of dishes, the menu generation system stores a sample dish image obtained by imaging the appearance of each dish, a dish name, and a dish attribute in association with each other. The cooking attributes include the type of dish (appetizer, salad, meat dish, fish dish, drink, etc.), the order in which dishes are served, the cooking genre (French cuisine, Sichuan cuisine, etc.), and the like.

  When the menu generation system receives the appearance image from the terminal device 2, the menu generation system identifies the dish name of each dish included in the received appearance image based on the received appearance image and the stored sample dish image and dish name.

  In the dish name specifying process, first, the menu generation system detects an image of a dish having a predetermined degree of coincidence with the sample dish image from the received appearance image using an image matching method such as template matching. Next, the menu generation system specifies the dish name of the sample dish image used for detecting the dish image, and associates the identified dish name with the detected dish image.

  Next, the menu generation system extracts dish attributes associated with all of the dish names specified based on the received appearance image. The menu generation system calculates the similarity between each of the extracted dish attributes and the dish attribute associated with another dish name. In the example shown in FIG. 1, “coffee” and “tea” are included in the dish names extracted by the menu generation system. In this case, the dish type “drink” associated with “coffee” and the order of food provision “fifth”, the dish type “drink” associated with “tea” and the dish provision order “fifth” 1 is calculated as the similarity. It should be noted that the dish type “bread rice” associated with “bread” and the order of food provision “third”, and the dish type “drink” associated with “coffee” and the order of food provision “fifth” 0 is calculated as the similarity to.

  The menu generation system, when the extracted dish attribute has a dish attribute having a predetermined degree of similarity (1 or more) with the dish attribute associated with another dish name, the dish attribute associated with another dish name. It is determined that the dish name is specified incorrectly. When the menu generation system determines that the specification of the dish name is incorrect, the menu generation system transmits to the terminal device 2 error specification notification information indicating that the specification of the dish name is incorrect.

  As described above, when the menu generation system specifies the dish name of each dish based on the appearance images of two or more dishes constituting the predetermined menu transmitted from the terminal device 2, It can be determined automatically. Accordingly, it is possible to provide a menu generation system, a menu generation method, and a control program that can improve the accuracy of the menu name determined from the appearance image obtained by photographing two or more items.

  The description of FIG. 1 described above is merely an explanation for deepening the understanding of the contents of the present invention. Specifically, the present invention may be implemented in various embodiments described below, and may be implemented in various modifications without substantially exceeding the principle of the present invention. All such variations are within the scope of the present disclosure and the specification.

(Configuration of menu generation system 1)
FIG. 2 is a diagram illustrating an example of a schematic configuration of the menu generation system 1.

  The menu generation system 1 includes a plurality of terminal devices 2 and a server 3. The terminal device 2 and the server 3 are connected to each other via a communication network. For example, the terminal device 2 and the server 3 are connected to each other via a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7.

  A program executed by the server 3 (for example, a search program) and a program executed by the terminal device 2 (for example, a browsing program) are performed using a communication protocol such as a hypertext transfer protocol (HTTP). Communicate.

(Schematic configuration of the terminal device 2)
FIG. 3 is a diagram illustrating an example of a schematic configuration of the terminal device 2.

  The terminal device 2 is connected to the server 3 via the base station 4, the mobile communication network 5, the gateway 6, and the Internet 7, and communicates with the server 3. The terminal device 2 requests the server 3 for a search instruction in accordance with the operation of the terminal operation unit 23 (button or the like) by the user. In addition, the terminal device 2 receives and displays display data related to a menu name identification screen or the like from the server 3. For this purpose, the terminal device 2 includes a terminal communication unit 21, a terminal storage unit 22, a terminal operation unit 23, a terminal display unit 24, a terminal imaging unit 25, and a terminal processing unit 26.

  In the present embodiment, a multifunctional mobile phone (so-called “smart phone”) is assumed as the terminal device 2, but the present invention is not limited to this. The terminal device 2 only needs to be applicable to the present invention. For example, a mobile phone (so-called “feature phone”), a portable information terminal (Personal Digital Assistant, PDA), a portable game machine, a portable music player, a tablet terminal, a tablet A PC (Personal Computer), a notebook PC, or the like may be used. The terminal device 2 may be a menu terminal or the like that is installed at a restaurant and a restaurant customer orders a menu.

  The terminal communication unit 21 includes a communication interface circuit including an antenna whose sensitivity band is a predetermined frequency band, and connects the terminal device 2 to the wireless communication network. The terminal communication unit 21 establishes a radio signal line by a CDMA (Code Division Multiple Access) method or the like with the base station 4 through a channel assigned by the base station 4, and communicates with the base station 4. Do. Then, the terminal communication unit 21 transmits the data supplied from the terminal processing unit 26 to the server 3 or the like. Further, the terminal communication unit 21 supplies the data received from the server 3 or the like to the terminal processing unit 26.

  The terminal storage unit 22 includes, for example, a semiconductor memory device. The terminal storage unit 22 stores an operating system program, a driver program, an application program, data, and the like used for processing in the terminal processing unit 26. For example, the terminal storage unit 22 stores, as a driver program, an input device driver program that controls the terminal operation unit 23, an output device driver program that controls the terminal display unit 24, and the like. In addition, the terminal storage unit 22 stores, as an application program, a program that acquires and displays display data related to menu information and the like. The terminal storage unit 22 stores, as data, a restaurant ID (Identification) of a restaurant related to the user, an appearance image, and the like. Further, the terminal storage unit 22 may temporarily store temporary data related to a predetermined process.

  The terminal operation unit 23 may be any device as long as the terminal device 2 can be operated, such as a touch panel or a key button. The user can input characters, numbers, symbols, and the like using the terminal operation unit 23. When operated by the user, the terminal operation unit 23 generates a signal corresponding to the operation. The generated signal is supplied to the terminal processing unit 26 as a user instruction.

  The terminal display unit 24 may be any device as long as it can display moving images, still images, and the like, for example, a touch panel display device, a liquid crystal display, an organic EL (Electro-Luminescence) display, and the like. The terminal display unit 24 displays a moving image corresponding to the moving image data supplied from the terminal processing unit 26, a still image corresponding to the still image data, and the like.

  The terminal imaging unit 25 includes an imaging optical system, an imaging element, an image processing unit, and the like. The imaging optical system is an optical lens, for example, and forms an image of a light beam from a subject on the imaging surface of the imaging device. The imaging element is a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and outputs an image of a subject image formed on the imaging surface. The image processing unit creates and outputs moving image data in a predetermined file format every predetermined period from images continuously generated by the image sensor. Alternatively, the image processing unit creates and outputs still image data in a predetermined file format from the image generated by the image sensor.

  The terminal processing unit 26 includes one or a plurality of processors and their peripheral circuits. The terminal processing unit 26 controls the overall operation of the terminal device 2 and is, for example, a CPU (Central Processing Unit). The terminal processing unit 26 is configured to execute the terminal communication unit 21 and the terminal so that various processes of the terminal device 2 are executed in an appropriate procedure based on a program stored in the terminal storage unit 22, an operation of the terminal operation unit 23, and the like. The operation of the display unit 24 and the like is controlled. The terminal processing unit 26 executes processing based on programs (operating system program, driver program, application program, etc.) stored in the terminal storage unit 22. Further, the terminal processing unit 26 can execute a plurality of programs (such as application programs) in parallel.

  The terminal processing unit 26 includes a terminal transmission unit 262, a display processing unit 261, and a terminal reception unit 263. Each of these units included in the terminal processing unit 26 is a functional module realized by a program executed by a processor included in the terminal processing unit 26. Alternatively, these units included in the terminal processing unit 26 may be implemented in the terminal device 2 as firmware.

  The display processing unit 261 acquires and displays display data and the like for displaying various screens such as an imaging screen and a specific screen described later. The display processing unit 261 creates drawing data based on the received display data. In other words, the display processing unit 261 analyzes the received display data to identify control data and content data, lays out the content data that is also identified based on the identified control data, and creates drawing data. Then, the display processing unit 261 outputs the created drawing data to the terminal display unit 24. The terminal transmission unit 262 transmits a search instruction to the server 3 via the terminal communication unit 21 in accordance with an instruction from the user. The terminal receiving unit 263 receives display data and the like for displaying a specific screen including the dish name specified based on the appearance image from the server 3 via the terminal communication unit 21.

(Display function of terminal device 2)
FIG. 4A is a diagram illustrating an example of a start screen 400 displayed on the terminal display unit 24 by the display function of the terminal device 2. On the start screen 400, an imaging start button 401, a dish registration button 402, and the like are displayed.

  The imaging start button 401 is an input button for starting the imaging function of the terminal device 2. When the imaging start button 401 is tapped by the user operating the terminal operation unit 23, the imaging function is started. When the imaging function is started, an imaging screen 410 that displays an image acquired by the terminal imaging unit 25 is displayed on the terminal display unit 24. A tap is an example of a designation operation by a user.

  The dish registration button 402 is an input button for registering in the server 3 a dish name, a dish attribute, and the like of a dish provided at a restaurant related to the user. When the imaging start button 401 is tapped by the user operating the terminal operation unit 23, a registration screen (not shown) is displayed on the terminal display unit 24. In the displayed registration screen, when a dish name, a dish attribute, and the like are input by an operation of the terminal operation unit 23 by the user, the input dish name, a dish attribute, and the like are transmitted to the server 3. Cooking attributes are registered.

  FIG. 4B is a diagram illustrating an example of the imaging screen 410 displayed on the terminal display unit 24 by the display function of the terminal device 2. On the imaging screen 410, an acquired image display area 411, an imaging button 412, and the like are displayed.

  The acquired image display area 411 is a display area in which an image acquired by the terminal imaging unit 25 is displayed. The acquired image display area 411 displays the acquired image every time the terminal imaging unit 25 acquires an image every predetermined time.

  The imaging button 412 is an input button for storing the image displayed in the acquired image display area 411 as an appearance image. When the imaging button 412 is tapped by the user operating the terminal operation unit 23, the image displayed in the acquired image display area 411 when the imaging button 412 is tapped is stored in the terminal storage unit 22 as an appearance image. Then, the terminal device 2 transmits the appearance image to the server 3 in order to cause the server 3 to identify the dish name of each dish included in the appearance image and to identify a predetermined menu configured by each identified dish. .

  FIG. 5 is a diagram illustrating an example of the specific screen 500 displayed on the terminal display unit 24 by the display function of the terminal device 2. The specific screen 500 displays specific dish information 501, a specified appearance image display button 502, a correction button 503, a registration button 504, and the like.

  In the specific dish information 501, a dish name, a dish attribute, and the like specified by a specific process described later are displayed so as to be selectable. The specific dish information 501 includes single specific dish information 501a and / or duplicate specific dish information 501b. The single specific dish information 501a is specific dish information having different dish attributes from the other specific dish information 501 among the specific dish information 501 including the dish name and the dish attribute specified by the specifying process. The duplicate specific dish information 501b is specific dish information having the same dish attribute as at least one of the other specific dish information 501 among the specific dish information 501 including the dish name and the dish attribute specified by the specifying process. is there.

  In the example shown in FIG. 5, for example, the dish attributes “appetizer” (type of dish) and “first” (the order in which dishes are provided) of the single specific dish information 501 a of the dish name “squid cold appetizer” are other. It is different from the cooking attributes of all the specific cooking information 501. On the other hand, the dish attributes “drink” (type of dish) and “fifth” (the order in which dishes are provided) of the duplicate specific dish information 501b with the dish name “coffee” are the dishes of the duplicate specific dish information 501b with the dish name “tea”. Same as attribute.

  As shown in FIG. 5, on the specific screen 500, the display modes of the single specific dish information 501a and the overlapped specific dish information 501b may be displayed so as to be different from each other. Thereby, it becomes possible to notify a user that the specification of the dish name of the duplication specific dish information 501b is incorrect. Note that the overlapping specific dish information 501b displayed in the specific screen 500 and having a display mode different from that of the single specific dish information 501a is an example of erroneous specification notification information.

  The specified appearance image display button 502 is an input button for displaying a specified appearance image screen 600 that displays an appearance image in which the specific dish information 501 displayed on the specification screen 500 is specified. When the specified appearance image display button 502 is tapped by the user operating the terminal operation unit 23, a specified appearance image screen 600 described later is displayed. Hereinafter, the appearance image in which the specific dish information 501 is specified may be referred to as a specified appearance image.

  The correction button 503 is an input button for displaying a correction screen 700 for correcting the specific dish information 501 displayed on the specific screen 500. For example, when any one of the specific dish information 501 and the correction button 503 are tapped by the operation of the terminal operation unit 23 by the user, a correction screen 700 for correcting the tapped specific dish information 501 is displayed on the terminal. It is displayed on the display unit 24.

  The registration button 504 is an input button for displaying a registration screen 900 for registering the dish name, dish attribute, and the like of the specific dish information 501 displayed on the specific screen 500. When the registration button 504 is tapped by the user operating the terminal operation unit 23, a registration screen 900 described later is displayed.

  FIG. 6A is a diagram illustrating an example of the specified appearance image screen 600 displayed on the terminal display unit 24 by the display function of the terminal device 2. On the specified appearance image screen 600, a single specific dish identification frame 601a and / or a duplicate specific dish identification frame 601b, a return button 602, a correction button 603, a registration button 604, and the like are displayed.

  The single specific dish identification frame 601a is a display frame showing an image of a dish corresponding to the single specific dish information 501a displayed on the specific screen 500. The duplicate specific dish identification frame 601b is a display frame that shows an image of a dish corresponding to the duplicate specific dish information 501b displayed on the specific screen 500. When any one of the duplicate specific dish identification frames 601b is designated by the operation of the terminal operation unit 23 by the user, an erroneous specification screen 610 is displayed.

  As shown in FIG. 6A, in the specified appearance image screen 600, the display modes of the single specific dish identification frame 601a and the duplicate specific dish identification frame 601b may be displayed differently from each other. Thereby, it is possible to notify the user that the specification of the dish name of the duplicate specific dish information corresponding to the duplicate specific dish identification frame 601b is incorrect. In addition, the overlapped specific dish identification frame 601b of the display mode different from the single specific dish identification frame 601a displayed on the identified external appearance image screen 600 is an example of erroneous specification notification information.

  The return button 602 is an input button for displaying the specific screen 500. When the return button 602 is tapped by the operation of the terminal operation unit 23 by the user, the display of the specified appearance image screen 600 is ended, and the specific screen 500 displayed before the specified appearance image screen 600 is displayed is displayed. Is displayed.

  The correction button 603 is an input button for displaying a correction screen 700 for correcting the dish name, dish attribute, and the like of the duplicate specific dish information 501b corresponding to the duplicate specific dish identification frame 601b displayed on the specified appearance image screen 600. is there. For example, when any one of the duplicate specific dish identification frame 601b and the correction button 603 are tapped by the operation of the terminal operation unit 23 by the user, a correction screen 700 described later is displayed on the terminal display unit 24.

  The registration button 604 is an input button for displaying a registration screen 900 for registering the dish name, dish attribute, and the like of the specific dish information 501 of each dish included in the appearance image displayed on the specified appearance image screen 600. When the registration button 604 is tapped by the user operating the terminal operation unit 23, a registration screen 900 described later is displayed.

  FIG. 6B is a diagram illustrating an example of an erroneous specification screen 610 displayed on the terminal display unit 24 by the display function of the terminal device 2. On the erroneous specification screen 610, duplicate specific dish information 611, a return button 612, a correction button 613, and the like are displayed. In the example shown in FIG. 6B, the erroneous specification screen 610 is displayed so as to be superimposed on the specified appearance image screen 600. However, after the display of the specified appearance image screen 600 is ended, the erroneous specification screen 610 is displayed. May be displayed.

  In the duplicate specific dish information 611, a dish name and a dish attribute corresponding to the duplicate specific dish identification frame 601b displayed on the identified appearance image screen 600 are displayed so as to be selectable. The duplicate specific dish information 611 is an example of erroneous specification notification information.

  The return button 612 is an input button for ending the display of the erroneous specification screen 610. When the return button 612 is tapped by the operation of the terminal operation unit 23 by the user, the display of the erroneous specification screen 610 ends.

  The correction button 613 is an input button for displaying a correction screen 700 for correcting the dish name, dish attribute, and the like of the duplicated specific dish information 611 displayed on the erroneous specification screen 610. For example, when any one of the duplicate specific dish information 611 and the correction button 613 are tapped by the user operating the terminal operation unit 23, the correction screen 700 is displayed on the terminal display unit 24.

  FIG. 7 is a diagram illustrating an example of a correction screen 700 displayed on the terminal display unit 24 by the display function of the terminal device 2. On the correction screen 700, correction specific dish information 701, candidate dish information 702, a return button 703, a correction registration button 704, and the like are displayed.

  The corrected specific dish information 701 includes the specific dish information 501 and the specific dish information corresponding to the duplicate specific dish identification frame 601b and the duplicate specific dish information that are tapped at the same time when any of the correction buttons 503, 603, and 613 is tapped. 611 cooking names, cooking attributes, and the like are displayed.

  In the candidate dish information 702, candidate dish information registered in place of the modified specific dish information 701 as a dish constituting a predetermined menu is displayed in a selectable manner. In the candidate dish information 702, a dish name, a dish attribute, and the like of the candidate dish information 702 are displayed. The candidate dish information 702 is, for example, a dish name and a dish attribute associated with a sample dish image similar to a dish image corresponding to the modified specific dish information 701 in the appearance image. Further, the candidate dish information 702 may be a dish name associated with a dish attribute that has not been identified and a predetermined dish attribute among the particular dish information 501 displayed on the particular screen 500. For example, in the example illustrated in FIG. 5, the dish information corresponding to the “second” “order of serving dishes” is not specified in the dish attribute of “the order of serving dishes”. In this case, the dish name and the dish attribute associated with the “second” dish provision order may be specified as the candidate dish information 702.

  The return button 703 is an input button for displaying any one of the specific screen 500, the specified appearance image screen 600, and the erroneous specification screen 610. When the return button 602 is tapped by the operation of the terminal operation unit 23 by the user, the display of the correction screen 700 is terminated, and the specific screen 500 and the specified appearance image screen that are displayed before the correction screen 700 is displayed. Either 600 or an erroneous specification screen 610 is displayed.

  The correction registration button 704 is an input button for performing a setting for correcting the selected (tapped) candidate dish information 702 to the corrected specific dish information 701 and displaying the specific screen 800 on the terminal display unit 24. For example, when any one of the corrected specific dish information 701 and the correction registration button 704 are tapped by the operation of the terminal operation unit 23 by the user, the corrected specific dish information 701 is used as a dish constituting a predetermined menu. It is set instead of the modified specific dish information 701. Then, the display of the correction screen 700 ends, and the specific screen 800 is displayed on the terminal display unit 24.

  FIG. 8 is a diagram illustrating an example of the specific screen 800 displayed on the terminal display unit 24 by the display function of the terminal device 2. The specific screen 800 displays specific dish information 801, a specified appearance image display button 802, a correction button 803, a registration button 804, and the like.

  FIG. 8 is a diagram illustrating an example of the specific screen 800 displayed on the terminal display unit 24 by the display function of the terminal device 2. The specific screen 800 displays specific dish information 801, a specified appearance image display button 802, a correction button 803, a registration button 804, and the like.

  In the specific dish information 801, the corrected specific dish information 701 is deleted from the specific dish information 501 shown in FIG. 5, and set candidate dish information 702 is displayed instead of the corrected specific dish information 701. The specified appearance image display button 802 is an input button for displaying a specified appearance image screen 600 that displays an appearance image in which the specific dish information 801 displayed on the specification screen 800 is specified. The correction button 803 is an input button for displaying a correction screen 700 for correcting the specific dish information 801 displayed on the specific screen 800. A registration button 804 is an input button for displaying a registration screen 900 for registering a dish name, dish attribute, and the like of the particular dish information 801 displayed on the particular screen 800. When the registration button 804 is tapped by the user operating the terminal operation unit 23, a registration screen 900 described later is displayed.

  FIG. 9 is a diagram illustrating an example of a registration screen 900 displayed on the terminal display unit 24 by the display function of the terminal device 2. On the registration screen 900, a menu name input box 901, a return button 902, a registration execution button 903, and the like are displayed.

  The menu name input box 901 is an input box for inputting a name of a predetermined menu. When a character string is input to the menu name input box 901 by the user operating the terminal operation unit 23, the character string input to the menu name input box 901 is displayed as the menu name.

  A return button 902 is an input button for displaying any one of the specific screen 500, the specified appearance image screen 600, and the specific screen 800. When the registration screen 900 is a screen displayed when the registration button 504 is tapped, when the return button 902 is tapped by the operation of the terminal operation unit 23 by the user, the display of the registration screen 900 is ended and the specific screen is displayed. A screen 500 is displayed. When the registration screen 900 is a screen displayed when the registration button 604 is tapped, when the return button 902 is tapped by the operation of the terminal operation unit 23 by the user, the display of the registration screen 900 is terminated. The specified appearance image screen 600 is displayed. When the registration screen 900 is a screen displayed when the registration button 804 is tapped, when the return button 902 is tapped by the operation of the terminal operation unit 23 by the user, the display of the registration screen 900 is terminated. The specific screen 800 is displayed.

  The registration execution button 903 is a menu name input in the menu name input box 901 together with the dish name and dish attribute in any of the dish information 501, the dish information corresponding to the duplicate dish identification frame 601 b, and the dish information 801. It is an input button for registering. When the registration screen 900 is a screen displayed when the registration button 504 is tapped, when the registration execution button 903 is tapped by the operation of the terminal operation unit 23 by the user, the dish name and the dish of the specific dish information 501 are displayed. Along with the attribute and the like, the menu name input in the menu name input box 901 is registered by the server. In addition, when the registration screen 900 is a screen displayed when the registration button 604 is tapped, when the registration execution button 903 is tapped by the user's operation of the terminal operation unit 23, the overlapping specific dish identification frame 601b is displayed. The menu name input in the menu name input box 901 is registered by the server together with the corresponding dish name and dish attribute. When the registration screen 900 is a screen displayed when the registration button 804 is tapped, when the registration execution button 903 is tapped by the operation of the terminal operation unit 23 by the user, the dish name of the specific dish information 801 is displayed. The menu name input in the menu name input box 901 is registered by the server together with the cooking attribute and the like.

(Schematic configuration of server 3)
FIG. 10 is a diagram illustrating an example of a schematic configuration of the server 3.

  For each of the plurality of dishes, the server 3 stores a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute in association with each other, and accepts an appearance image including the appearance of each dish that constitutes a predetermined menu. The dish name of each dish included in the appearance image is specified based on the image and the stored sample dish image and dish name. In order to realize the functions as described above, the server 3 includes a server communication unit 31, a server storage unit 32, and a server processing unit 33.

  The server communication unit 31 has a communication interface circuit for connecting the server 3 to the Internet 7. The server communication unit 31 supplies data received from the terminal device 2 and the like to the server processing unit 33. In addition, the server communication unit 31 transmits the data supplied from the server processing unit 33 to the terminal device 2 or the like.

  The server storage unit 32 includes, for example, at least one of a semiconductor memory, a magnetic disk device, and an optical disk device. The server storage unit 32 stores a driver program, an operating system program, an application program, data, and the like used for processing by the server processing unit 33. For example, the server storage unit 32 stores a communication device driver program for controlling the server communication unit 31 and the like as a driver program. Various programs may be installed in the server storage unit 32 using a known setup program or the like from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM. The server storage unit 32 stores a table, a table, and the like which will be described later as data. Further, the server storage unit 32 may temporarily store temporary data related to a predetermined process.

  The table in the present embodiment is an example of a database format, and may be a format in which data is pooled in the server.

  The server processing unit 33 includes one or a plurality of processors and their peripheral circuits. The server processing unit 33 comprehensively controls the overall operation of the server 3 and is, for example, a CPU. The server processing unit 33 controls the operation of the server communication unit 31 and the like so that various processes of the server 3 are executed in an appropriate procedure based on a program or the like stored in the server storage unit 32. The server processing unit 33 executes processing based on programs (operating system program, driver program, application program, etc.) stored in the server storage unit 32. Further, the server processing unit 33 can execute a plurality of programs (such as application programs) in parallel.

  The server processing unit 33 includes a server reception unit 331, a specification unit 332, a calculation unit 333, and a server transmission unit 334. Each of these units included in the server processing unit 33 is a functional module implemented by a program executed on a processor included in the server processing unit 33. Alternatively, each of these units included in the server processing unit 33 may be mounted on the server 3 as an independent integrated circuit, a microprocessor, or firmware.

(Data structure of various tables)
FIG. 11 is a diagram illustrating an example of the data structure of various tables. The server storage unit 32 does not store all or some of the various tables described below, and one or a plurality of other devices (such as a database device) different from the server 3 Or you may memorize a part. In this case, when using the information stored in these various tables, the server 3 accesses another device that stores the table and acquires the information stored in the table. Further, each piece of information included in each table may be stored in a format other than the table, for example, may be stored independently.

  FIG. 11A shows a dish management table. In the dish management table, a restaurant ID, dish information, and the like are stored in association with each other for each restaurant.

  The restaurant ID is identification information for identifying each restaurant. In the dish information, a dish information ID, dish name, type, order of provision, dish genre, sample dish image, and the like are stored in association with each other. Note that a plurality of pieces of cooking information may be stored in association with each restaurant ID. The sample dish image stores the file name of the sample dish image.

  FIG. 11B shows a menu management table. In the menu management table, a restaurant ID, menu information, and the like are stored in association with each other for each restaurant.

  In the menu information, a menu ID, an appearance image, a menu name, specific dish information, and the like are associated and stored. The menu ID is identification information for identifying each menu. In the appearance image, the file name of the appearance image received from the terminal device 2 is stored. The menu name is a name of a predetermined menu configured by each dish included in the appearance image. As the specific dish information, the dish information ID and the frame information of the dish information specified by the specifying process are stored. A plurality of specific dish information is stored for each menu information. In addition, when a plurality of pieces of specific dish information are stored, each piece of specific dish information is stored according to the order in which each piece of specific dish information is provided. The frame information is a display position on the appearance image of the single specific dish identification frame 601a and / or the duplicate specific dish identification frame 601b.

(Operation sequence of menu generation system 1)
FIG. 12 is a diagram illustrating an example of an operation sequence of the menu generation system 1. This operation sequence cooperates with each element of the terminal device 2 and the server 3 mainly by the terminal processing unit 26 and the server processing unit 33 based on the programs stored in the terminal storage unit 22 and the server storage unit 32 in advance. And executed.

  First, when the display processing unit 261 of the terminal device 2 detects that the imaging start button 401 displayed on the start screen 400 is selected (tapped) by the operation of the terminal operation unit 23 by the user, the display processing unit 261 displays the imaging screen 410. It is displayed on the terminal display unit 24. Next, when the display processing unit 261 detects that the imaging button 412 displayed on the start screen 400 is selected by the operation of the terminal operation unit 23 by the user, the display processing unit 261 displays the image displayed in the acquired image display area 411. It memorize | stores in the terminal memory | storage part 22 as an external appearance image (step S101).

  Next, the display processing unit 261 reads out the appearance image and the restaurant ID stored in the terminal storage unit 22 and passes them to the terminal transmission unit 262, and instructs the terminal transmission unit 262 to transmit the appearance image. When the display processing unit 261 is instructed to transmit the appearance image, the terminal transmission unit 262 sends a specific request including the appearance image and the restaurant ID received from the display processing unit 261 to the server 3 via the terminal communication unit 21. Transmit (step S102).

  Next, when the server receiving unit 331 of the server 3 receives the specific request from the terminal device 2 via the server communication unit 31, it passes the appearance image and the restaurant ID included in the specific request to the specifying unit 332, The specifying unit 332 is instructed to execute the specifying process. The server reception unit 331 that receives a specific request including an appearance image is an example of an input processing unit that receives an appearance image.

  The identification unit 332 executes the identification process when instructed to execute the identification process from the server reception unit 331 (step S103). Details of the specific processing will be described later.

  Next, the server transmission unit 334 transmits display data for displaying the specific screen 500 created by the specific process to the terminal device 2 via the server communication unit 31 (step S104).

  Next, when receiving the display data from the server 3 via the terminal communication unit 21, the terminal receiving unit 263 of the terminal device 2 passes the display data to the display processing unit 261, and executes the screen display process to the display processing unit 261. Instruct. The display processing unit 261 displays the specific screen 500 on the terminal display unit 24 based on the display data when the terminal receiving unit 263 is instructed to execute the screen display process.

  Next, when the display processing unit 261 of the terminal device 2 detects that any of the various buttons displayed on the specific screen 500 is selected (tapped) by the operation of the terminal operation unit 23 by the user (step S105). The terminal transmission unit 262 transmits a registration request to the server 3 via the terminal communication unit 21 (step S106). That is, when the display processing unit 261 detects selection of the specified appearance image display button 502, the terminal transmission unit 262 transmits the specified appearance image screen request to the server 3 as a registration request. When the display processing unit 261 detects selection of the correction button 503, the terminal transmission unit 262 transmits a correction screen request to the server 3 as a registration request. When the display processing unit 261 detects selection of the registration button 504, the terminal transmission unit 262 transmits a registration screen request to the server 3 as a registration request.

  Next, when receiving a registration request from the terminal device 2 via the server communication unit 31, the server reception unit 331 of the server 3 passes the registration request to the specification unit 332 and instructs the specification unit 332 to execute the registration process. . When the specifying unit 332 is instructed to execute the registration process from the server reception unit 331, the specifying unit 332 executes the registration process (step S107). Details of the registration process will be described later.

  Next, the server transmission unit 334 transmits display data for displaying various screens created by the specifying process to the terminal device 2 via the server communication unit 31 (step S108). The various screens are the specified appearance image screen 600, the correction screen 700, or the registration screen 900.

  Next, when receiving the display data from the server 3 via the terminal communication unit 21, the terminal receiving unit 263 of the terminal device 2 passes the display data to the display processing unit 261, and executes the screen display process to the display processing unit 261. Instruct. The display processing unit 261 displays various screens on the terminal display unit 24 based on the display data when the terminal receiving unit 263 is instructed to execute the screen display process.

  Next, when the display processing unit 261 of the terminal device 2 detects that any of the various buttons displayed on the various screens is selected (tapped) by the operation of the terminal operation unit 23 by the user (Step S109). The terminal transmission unit 262 transmits a registration request to the server 3 via the terminal communication unit 21 (step S110). That is, when the display processing unit 261 detects selection of the specified appearance image display button 502, the terminal transmission unit 262 transmits the specified appearance image screen request to the server 3 as a registration request. When the display processing unit 261 detects selection of the duplicate specific dish identification frame 601b, the terminal transmission unit 262 transmits an erroneous specification screen request to the server 3 as a registration request. Further, when the display processing unit 261 detects selection of the correction button 503, the correction button 603, the correction button 613, or the correction button 803, the terminal transmission unit 262 transmits a correction screen request to the server 3 as a registration request. When the display processing unit 261 detects selection of the correction registration button 704, the terminal transmission unit 262 transmits the corrected specific screen request to the server 3 as a registration request. When the display processing unit 261 detects selection of the registration button 504, the registration button 604, or the registration button 804, the terminal transmission unit 262 transmits a registration screen request to the server 3 as a registration request. When the display processing unit 261 detects selection of the registration execution button 903, the terminal transmission unit 262 transmits an information registration request to the server 3 as a registration request. The information registration request includes the menu name input in the menu name input box 901 and the menu ID related to the specific result. When the registration screen 900 is a screen displayed when the registration button 504 is tapped, the dish information ID, dish name, dish attribute, and the like of the specific dish information 501 are included in the information registration request. When the registration screen 900 is a screen displayed when the registration button 604 is tapped, the dish information ID, the dish name, the dish attribute, and the like corresponding to the duplicate specific dish identification frame 601b are included in the information registration request. It is. When the registration screen 900 is a screen displayed when the registration button 804 is tapped, the dish information ID, the dish name, the dish attribute, and the like of the specific dish information 801 are included in the information registration request.

  Next, when the server receiving unit 331 of the server 3 receives various registration requests from the terminal device 2 via the server communication unit 31, it passes the registration request to the specifying unit 332 and instructs the specifying unit 332 to execute the registration process. To do. When the specifying unit 332 is instructed to execute the registration process from the server receiving unit 331, the specifying unit 332 executes the registration process (step S111). Details of the registration process will be described later.

  Next, when the display data for displaying various screens is created by the specific process, the server transmission unit 334 transmits the created display data to the terminal device 2 via the server communication unit 31 (step S112). ). The various screens are the specified appearance image screen 600, the erroneous specification screen 610, the correction screen 700, the specific screen 800, or the registration screen 900.

  Thereafter, step S109 to step S112 are executed according to the operation of the terminal operation unit 23 by the user.

(Specific processing)
FIG. 13 is a flowchart illustrating an example of the specifying process. The specifying process shown in FIG. 14 is executed in step S103 of FIG.

  First, the specifying unit 332 detects a dish image having a predetermined degree of coincidence with the sample dish image from the appearance image received from the server reception unit 331 using an image matching method such as template matching (step S201). That is, the specifying unit 332 calculates the degree of coincidence of the image of the portion that overlaps the template in the appearance image by using part or all of the sample dish image as a template. The degree of coincidence is SSD (Sum of Squared Difference) or SAD (Sum of Absolute Difference). Next, the specifying unit 332 associates the restaurant ID received from the server receiving unit 331 with the newly added menu ID, the file name of the appearance image, and the sample dish image used to detect the image of the dish. The associated dish information ID, the detected dish information ID of the dish, and the display position (frame information) of the dish image are stored in the menu management table (step S202).

  Next, the identifying unit 332 refers to the dish management table and extracts dish attributes associated with all the dish information IDs identified based on the received appearance image. Hereinafter, the dish information of the extracted dish information ID may be referred to as specific dish information. The specifying unit 332 determines whether there is a cooking attribute that is the same as the cooking attribute of the other specific cooking information among the extracted cooking attributes (step S203). In this determination process, first, the specifying unit 332 passes the extracted dish attribute to the calculation unit 333. For each of the plurality of received cooking attributes, the calculation unit 333 sets the similarity to 1 if each cooking attribute is the same as the other cooking attributes received, and if each cooking attribute is different from the other cooking attributes received, The similarity is calculated as 0, and the calculation result is passed to the specifying unit 332. When there is a cooking attribute having a similarity of a predetermined degree (1) or more, the specifying unit 332 has a cooking attribute that is the same (overlapping) as the cooking attribute of other specific cooking information in each of the extracted cooking attributes. Is determined.

  When there is a cooking attribute that is the same (overlapping) as the cooking attribute of the other specific cooking information among the extracted cooking attributes, the specification unit 332 specifies the cooking name in error. Is determined. Next, the specifying unit 332 displays the specifying screen 500 with the specific cooking information of the same cooking attribute (duplicating cooking attribute) as the overlapping specific cooking information 501b and the other specific cooking information as the single specific cooking information 501a. Display data is created (step S204), and the specifying process is terminated.

  In addition, when there is no cooking attribute that is the same as the cooking attribute of other specific cooking information among the extracted cooking attributes (step S203-No), the specifying unit 332 converts all the specific cooking information into a single specific cooking information. Display information for displaying the specific screen 500 is created as information 501a (step S205), and the specific process is terminated.

  The display data for displaying the specific screen 500 is created for each specific dish information 501 based on the dish information ID, dish name, and dish attribute extracted from the dish management table. The display data includes the menu ID of the menu displayed on the specific screen 500.

(registration process)
FIG. 14 is a flowchart illustrating an example of the registration process. The registration process shown in FIG. 14 is executed in steps S107 and S111 in FIG.

  First, the specifying unit 332 receives any of the registration request received from the server receiving unit 331 as a specified appearance image screen request, an erroneous specification screen request, a correction screen request, a corrected specific screen request, a registration screen request, and an information registration request. (Step S301).

  Next, the specifying unit 332 determines whether or not the specified registration request is a specified appearance image screen request (step S302). When determining that the registration request is a specified appearance image screen request (step S302—Yes), the specifying unit 332 creates display data for displaying the specified appearance image screen 600 (step S303), and performs registration processing. Exit. The display data for displaying the specified appearance image screen 600 is created based on the dish information ID of the dish and the display position (frame information) of the dish image extracted from the menu management table.

  Further, when determining that the registration request is not a specified appearance image screen request (No in step S302), the specifying unit 332 determines whether the registration request is an erroneous specification screen request (step S304).

  If the specifying unit 332 determines that the registration request is an erroneous specification screen request (step S304—Yes), the specification unit 332 creates display data for displaying the erroneous specification screen 610 (step S305), and ends the registration process.

  Further, when determining that the registration request is not an erroneous specification screen request (No in step S304), the specifying unit 332 determines whether the registration request is a correction screen request (step S306).

  When determining that the registration request is a correction screen request (step S306-Yes), the specifying unit 332 creates display data for displaying the correction screen 700 (step S307), and ends the registration process. The display data for displaying the correction screen 700 is created based on the candidate dish information 702 extracted based on the dish information ID, dish name, and dish attribute extracted from the dish management table.

  Further, when determining that the registration request is not a correction screen request (step S306-No), the specifying unit 332 determines whether the registration request is a corrected specific screen request (step S308).

  When determining that the registration request is a corrected specific screen request (step S308-Yes), the specifying unit 332 creates display data for displaying the specific screen 800 (step S309), and ends the registration process. . Note that the display data for displaying the specific screen 800 is the dish information ID, the dish name, the dish attribute, and the candidate dish information 702 set in place of the modified specific dish information 701 extracted from the dish management table. Created based on.

  Further, when determining that the registration request is not a corrected specific screen request (step S308-No), the specifying unit 332 determines whether the registration request is a registration screen request (step S310).

  When determining that the registration request is a registration screen request (step S310—Yes), the specifying unit 332 creates display data for displaying the registration screen 900 (step S311), and ends the registration process.

  Further, when determining that the registration request is an information registration request (step S310-No), the specifying unit 332 executes an information registration process (step S312), and ends the registration process. In the information registration process, the specifying unit 332 extracts the menu ID, menu name, dish information ID, dish name, and dish attribute included in the registration request, and associates the menu name and dish information ID with the extracted menu ID. Are stored in the menu management table.

  As described above in detail, the menu generation system 1 determines the dish name of each dish when identifying the dish name of each dish based on the appearance images of two or more dishes constituting the predetermined menu transmitted from the terminal device 2. Specific errors can be automatically determined. Accordingly, it is possible to provide a menu generation system, a menu generation method, and a control program that can improve the accuracy of the menu name determined from the appearance image obtained by photographing two or more items.

  Each display data described above includes a menu ID of a menu displayed on various screens.

(Modification 1)
The terminal device 2 is not limited to a communication device that communicates with the server 3 via the mobile communication network 5, but an access point (not shown) such as a stationary personal computer (PC), for example. An information terminal that communicates with the server 3 via the router (not shown) or the Internet may be used.

  FIG. 15 is a diagram illustrating another example of the schematic configuration of the menu generation system 10. In FIG. 15, the same components as those of the menu generation system 1 shown in FIG.

  The menu generation system 10 includes a plurality of terminal devices 2 and 20 and a server 3. The terminal device 20 is an information terminal such as a stationary personal computer (PC). The terminal device 20 has the same function as the terminal device 2. The terminal device 20 and the server 3 are connected to each other via the Internet 7. A program (for example, a search program) executed by the server 3 and a program (for example, a browsing program) executed by the terminal device 20 use a communication protocol such as a hypertext transfer protocol (HTTP). Communicate.

(Modification 2)
The menu generation systems 1 and 10 may be configured to include only the terminal devices 2 and 20. In this case, the terminal devices 2 and 20 have the functions of the server 3. The menu generation systems 1 and 10 may be configured to include only the server 3. In this case, the server 3 has the functions of the terminal devices 2 and 20.

(Modification 3)
The server processing unit 33 of the server 3 may include a generation unit 334 that generates menu display information in response to a menu information request from the terminal devices 2 and 20. In this case, when the menu information request from the terminal devices 2 and 20 is received by the server reception unit 331, the generation unit 334 extracts the menu ID included in the menu information request. Next, the generation unit 334 extracts the menu name and specific dish information associated with the menu ID from the menu management table. Next, the generation unit 334 generates display information of a menu in which specific dish information (dish name, dish attribute, etc.) is arranged according to the provision order of the specific dish information based on the extracted menu name and specific dish information. To do. Then, the server transmission unit 334 transmits the generated menu display information to the terminal devices 2 and 20 that transmitted the menu information request.

(Modification 4)
Further, the specifying unit 332 respecifies the dish name of each dish included in the appearance image so that the dish attribute associated with the dish name of the specific dish information is not duplicated based on the similarity calculated by the calculation unit 333. May be. In other words, when the extracted cooking attribute includes a cooking attribute that is the same (overlapping) as the cooking attribute of other specific cooking information, in step S204, the specifying unit 332 displays the specific cooking information of the overlapping cooking attribute. A process of changing to a dish name with another dish attribute may be executed. For example, first, the specifying unit 332 uses the image matching method to detect the specific dish information having the highest degree of coincidence calculated for the corresponding sample dish image from among the dish images of the specific dish information having the same dish attribute. To do. Next, the specifying unit 332 has the highest degree of coincidence among sample dish images corresponding to other dish attributes different from the overlapping dish attributes with respect to the dish image other than the highest detected specific dish information. Detect sample food images. Next, the specifying unit 332 stores the dish information of the detected sample dish image in the menu management table in association with the menu information of the captured predetermined menu. And the specific | specification part 332 produces the display data for displaying the specific screen 500 in which all the specific dish information is the single specific dish information 501a, and complete | finishes a specific process. The display data for displaying the specific screen 500 in which the specific dish information is all the single specific dish information 501a is an example of name suggestion information. Moreover, misidentification notification information and name proposal information are examples of notification information.

(Modification 5)
In the dish management table, the dish name, type, dish genre, and the like written in a plurality of languages may be stored for each dish information. In this case, in step S204, the specific screen 500 on which the single specific dish information 501a and the duplicate specific dish information 501b including the dish name, type, and / or dish genre written in each of a plurality of languages is displayed. Create display data to display. For example, when a dish name written in a first language (such as Japanese) and a dish name written in a second language (such as English) are stored in the dish management table in association with each other, written in the first language Single specific dish information 501a and duplicate specific dish information 501b including the prepared dish name and the dish name written in the second language are generated.

(Modification 6)
Further, the specifying unit 332 may execute a process of determining whether or not each of the extracted dish attributes is associated with a predetermined dish attribute in Step S203. For example, when the specific request transmitted by the terminal devices 2 and 20 includes a cooking genre together with the appearance image and the restaurant ID, the specifying unit 332 is associated with a predetermined cooking genre among the specified specific cooking information. If there is specific food information that has not been received, it is determined that the specification of the food name is incorrect. In this determination process, the specifying unit 332 first passes the cooking genres of all the specific cooking information to the calculating unit 333. The calculation unit 333 calculates the similarity as 1 if each of the received plurality of food genres is the same as the predetermined food genre, and calculates the similarity as 0 if they are different from the predetermined food genre. It passes to the specific part 332. When the similarity is equal to or greater than the predetermined degree (1), the specifying unit 332 determines that the specific dish information of the dish attribute is specific dish information associated with a predetermined dish genre, and the similarity is the predetermined degree. If it is less than (1), it is determined that the specific dish information of the dish attribute is specific dish information that is not associated with a predetermined dish genre.

  Thus, for example, when a user provides course dishes of Sichuan cuisine, etc., when specific dish information such as a dish name associated with a cuisine genre other than Sichuan cuisine is specified, an error in specifying the dish name is automatically performed. Can be determined automatically. Therefore, it is possible to improve the accuracy of the menu name determined from the appearance image obtained by photographing two or more dishes. The predetermined dish genre is an example of a predetermined dish attribute.

  In the sixth modification, the predetermined dish attribute may be the most dish attribute among the specified dish attributes. Thereby, even if the terminal devices 2 and 20 do not input a dish genre, a dish name specific error can be automatically determined.

  In step S203, a process for determining whether or not each of the extracted dish attributes is associated with a predetermined dish attribute is performed, and the identified dish information is associated with the transmitted dish genre. When it is determined that there is not the specific dish information, the specifying unit 332 transmits the dish name of the specific dish information that is not associated with the transmitted dish genre among the specified specific dish information. You may change to the other dish name linked | related with the genre. For example, the specifying unit 332 outputs a sample dish image having a second predetermined similarity or higher that is lower than a predetermined similarity with respect to each dish image of the specific dish information that is not associated with the transmitted cooking genre. To detect. Next, when the sample dish image is detected, the specifying unit, if the dish genre of the detected sample dish image is the same as the transmitted dish genre, the sample dish having a second predetermined similarity or higher. The dish information ID of the specific dish information in which the image is detected is changed to the dish information ID associated with the sample dish image used for detecting the dish image, and stored in the menu management table. And the specific | specification part 332 produces the display data for displaying the specific screen 500 with which specific dish information was detected, and complete | finishes a specific process. The display data for displaying the specific screen 500 in which the specific dish information is detected is an example of name suggestion information. The erroneous specification notification information and the name suggestion information are examples of specific information.

  It should be understood by those skilled in the art that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

DESCRIPTION OF SYMBOLS 1,10 Menu generation system 2,20 Terminal device 21 Terminal communication part 22 Terminal memory | storage part 23 Terminal operation part 24 Terminal display part 25 Terminal imaging part 26 Terminal processing part 261 Display processing part 262 Terminal transmission part 263 Terminal reception part 3 Server 31 Server communication unit 32 Server storage unit 33 Server processing unit 331 Server reception unit 332 Identification unit 333 Calculation unit 334 Server transmission unit 334 Generation unit 4 Base station 5 Mobile communication network 6 Gateway 7 Internet

Claims (20)

A menu generation system for generating a restaurant menu,
For each of a plurality of dishes, a storage unit that stores a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute in association with each other;
An input processing unit that accepts an appearance image including the appearance of each dish for a predetermined menu composed of two or more dishes of the plurality of dishes;
Based on the received appearance image and the stored sample dish image and dish name, a specifying unit that identifies a dish name of each dish included in the received appearance image;
For the dish name of each identified dish, a calculation unit that calculates the similarity between the dish attribute associated with the dish name of each dish and the dish attribute associated with the dish name of another dish;
A menu generation system comprising:   The identification unit, based on the similarity calculated by the calculation unit, the dish name of each dish included in the received appearance image so that the dish attribute associated with the dish name of each identified dish does not overlap. The menu generation system according to claim 1, wherein the menu is re-specified. The specific part is:
Calculating the degree of coincidence between the image of each dish included in the received appearance image and the sample dish image;
When dish attributes associated with the dish names of the identified dishes are duplicated, the degree of coincidence calculated for the corresponding sample dish image is the highest among the dish names of dishes with duplicate dish attributes. The menu generation system according to claim 2, wherein a dish name other than the dish name is changed.   The menu generation system according to any one of claims 1 to 3, wherein the dish attribute is a serving order in a menu including two or more dishes.   5. The menu generation system according to claim 4, further comprising: a generation unit that generates display information of a menu in which dish names of the identified dishes are arranged according to a provision order associated with each dish.   The menu generation system according to any one of claims 1 to 3, wherein the dish attribute is a kind of dish.   The transmission unit according to any one of claims 1 to 6, further comprising: a notification unit that notifies the user that there is an error in specifying the dish name by the specifying unit when the similarity is greater than or equal to a predetermined level. The menu generation system described.   The notification information includes at least one dish name of dish names associated with dish attributes having a similarity degree equal to or higher than the predetermined degree among the dish names of the identified dishes, and a similarity degree equal to or higher than the predetermined degree. The menu generation system according to claim 7, comprising name suggestion information changed to a dish name associated with another dish attribute different from the dish attribute. The storage unit stores a dish name written in a first language and a dish name written in a second language in association with each other for each of the plurality of dishes.
The menu generation system according to any one of claims 1 to 8, further comprising a transmission unit that transmits a dish name written in a first language of each dish or a dish name written in a second language to the terminal. A menu generation system for generating a restaurant menu,
For each of a plurality of dishes, a storage unit that stores a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute in association with each other;
An input processing unit that accepts an appearance image including the appearance of each dish for a predetermined menu composed of two or more dishes of the plurality of dishes;
Based on the received appearance image and the stored sample dish image and dish name, a specifying unit that identifies a dish name of each dish included in the received appearance image;
For the dish name of each identified dish, a calculation unit that calculates the similarity between the dish attribute associated with the dish name of each dish and a predetermined dish attribute;
A menu generation system comprising:   Based on the similarity calculated by the calculation unit, the specifying unit is configured such that all of the dish attributes associated with the dish names of the identified dishes become dish attributes associated with the predetermined dish attribute. The menu generation system according to claim 10, wherein a dish name of each dish included in the received appearance image is re-specified.   The menu generation system according to claim 10 or 11, wherein the specifying unit sets, as the predetermined dish attribute, the largest number of dish attributes among the dish attributes associated with the dish names of the identified dishes.   The menu generation system according to any one of claims 10 to 12, wherein the dish attribute is a dish genre. The storage unit stores a dish name written in a first language and a dish name written in a second language in association with each other for each of the plurality of dishes.
The menu generation system according to any one of claims 10 to 13, further comprising: a transmission unit configured to transmit a dish name written in the first language of each dish or a dish name written in the second language to the terminal.   15. The transmission unit according to claim 10, further comprising a transmission unit that notifies notification information indicating that there is an error in specifying the dish name by the specifying unit when the similarity is less than a predetermined level. Menu generation system.   The notification information includes name suggestion information obtained by changing a dish name that is not associated with the predetermined dish attribute among dish names of the identified dishes to a dish name associated with the predetermined dish attribute. The menu generation system according to claim 15. A menu generation method by a computer having a storage unit and generating a restaurant menu,
For each of a plurality of dishes, a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute are associated with each other and stored in the storage unit,
Accepting an appearance image including the appearance of each dish for a predetermined menu composed of two or more dishes of the plurality of dishes,
Based on the received appearance image and the stored sample dish image and dish name, the dish name of each dish included in the received appearance image is specified,
For the dish name of each identified dish, the similarity between the dish attribute associated with the dish name of each dish and the dish attribute associated with the dish name of another dish is calculated.
A menu generation method characterized by including: A computer control program that has a storage unit and generates a restaurant menu.
For each of a plurality of dishes, a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute are associated with each other and stored in the storage unit,
Accepting an appearance image including the appearance of each dish for a predetermined menu composed of two or more dishes of the plurality of dishes,
Based on the received appearance image and the stored sample dish image and dish name, the dish name of each dish included in the received appearance image is specified,
For the dish name of each identified dish, the similarity between the dish attribute associated with the dish name of each dish and the dish attribute associated with the dish name of another dish is calculated.
A control program that makes things happen. A menu generation method by a computer having a storage unit and generating a restaurant menu,
For each of a plurality of dishes, a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute are associated with each other and stored in the storage unit,
Accepting an appearance image including the appearance of each dish for a predetermined menu composed of two or more dishes of the plurality of dishes,
Based on the received appearance image and the stored sample dish image and dish name, the dish name of each dish included in the received appearance image is specified,
For the dish name of each identified dish, the similarity between the dish attribute associated with the dish name of each dish and a predetermined dish attribute is calculated.
A menu generation method characterized by including: A computer control program that has a storage unit and generates a restaurant menu.
For each of a plurality of dishes, a sample dish image obtained by imaging the appearance, a dish name, and a dish attribute are associated with each other and stored in the storage unit,
Accepting an appearance image including the appearance of each dish for a predetermined menu composed of two or more dishes of the plurality of dishes,
Based on the received appearance image and the stored sample dish image and dish name, the dish name of each dish included in the received appearance image is specified,
For the dish name of each identified dish, the similarity between the dish attribute associated with the dish name of each dish and a predetermined dish attribute is calculated.
A control program that makes things happen.

Images