JP2010061383A - Automatic recipe creation apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problems wherein doneness of dishes prepared by recipes cannot be evaluated objectively and past evaluations cannot be reflected in later recipes. <P>SOLUTION: An automatic recipe creation apparatus includes an imaging means for capturing a dish presentation image, an image processing means for extracting presentation features from the dish presentation image captured by the imaging means, a recipe database for storing presentation images and presentation standards of each kind of dish, and an evaluation means for comparing the presentation features extracted by the image processing means with the presentation standards stored in the recipe database. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an automatic recipe creation apparatus and program for evaluating the performance of a dish.

2. Description of the Related Art Conventionally, a recipe providing system in which a computer displays a recipe (such as a cooking name and a cooking method) using the food based on manually input food information has been considered (see, for example, Patent Document 1). On the other hand, with the widespread use of digital cameras that can be easily photographed, an increasing number of people shoot food and register it in blogs, or publish recipes, and there are Internet sites that evaluate the performance of dishes. In addition, with the improvement of image processing technology of digital cameras, various image recognition can be performed from captured images, and face recognition and facial expression recognition have been put into practical use.
Japanese Patent Application Laid-Open No. 11-066170

  However, in the past, it was not possible to objectively evaluate the performance of a dish prepared by looking at the recipe. Even if the taste of the dish cooked by looking at the recipe is too light or too dark, the next time you make the same dish, the same amount of recipe will be displayed as before, so the person who cooks will There was a problem that it would be the same seasoning again unless you remember the failure.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an automatic recipe creation apparatus and program capable of automatically evaluating the workmanship from a dish arrangement image without performing complicated operations and reflecting the evaluation result in the recipe for the next dish. Is to provide.

  An automatic recipe creation device according to the present invention includes an imaging unit that captures a dish arrangement image, an image processing unit that extracts a feature amount of the dish from the dish arrangement image captured by the imaging unit, and a type of dish. It has a recipe database for storing the cooking image of the dish and the serving standard, and an evaluation unit for comparing the serving feature amount extracted by the image processing unit with the serving standard stored in the recipe database. To do.

  Preferably, the serving feature amount extracted by the image processing means is determined by at least one of the color scheme of the dish, the arrangement and shape of the dish relative to the dish, the area ratio of the dish to the dish, and the dispersion of ingredients, The serving standard stored in the recipe database is determined by at least one of the color scheme of the dish, the arrangement and shape of the dish with respect to the dish, the area ratio of the dish to the dish, and the distribution of ingredients.

  Preferably, display means is further provided, and the evaluation means displays the evaluation result on the display means.

  Preferably, the evaluation unit displays the evaluation result on the display unit with a numerical value or a symbol.

  Preferably, the evaluation unit stores the dish arrangement image photographed by the imaging unit in association with the type of the dish in the recipe database as a past example.

  Preferably, the evaluation unit adds the evaluation result of the evaluation unit when storing the dish arrangement image photographed by the imaging unit in association with the type of the dish in the recipe database as a past example. It is memorized.

  Preferably, comment input means for inputting a comment is further provided, and the evaluation means stores, as a past example, the dish arrangement image taken by the imaging means in association with the type of the dish in the recipe database. Further, the evaluation result of the evaluation means and the information input by the comment input means are added and stored.

  Preferably, a recipe changing means is further provided for changing the contents of the recipe stored in the recipe database in accordance with the information input by the comment input means.

  The automatic recipe creation program according to the present invention is a program processed using a computer, and features an image input means for inputting a dish image and a dish feature from the dish image input by the image input means. The image processing procedure for extracting the amount and the serving standard stored in the recipe database for storing the cooking image and the serving standard for each type of cooking are compared with the serving feature amount extracted by the image processing means. And an evaluation procedure.

  In the present invention, the workmanship can be automatically evaluated from the dish arrangement image without performing complicated operations, and the evaluation result can be reflected in the recipe for the next dish.

  Embodiments relating to an automatic recipe creating apparatus and a program according to the present invention will be described below.

(First embodiment)
The first embodiment is an example in which an automatic recipe creation apparatus is realized by the digital camera 101. FIG. 1 is a block diagram illustrating a configuration of the digital camera 101. The digital camera 101 includes a photographing optical system 102, an image sensor 103, an AFE (analog front end) 104, an A / D conversion unit 105, an image buffer 106, a control unit 107, a display unit 108, and a memory card. An I / F 109, an operation unit 110, a memory 111, a date module 112, and a camera shake sensor 114 are included.

  In FIG. 1, the subject light incident on the photographing optical system 102 is incident on the light receiving surface of the image sensor 103. Here, the photographing optical system 102 includes a plurality of lenses such as a zoom lens and a focus lens, and moves the zoom lens according to a zoom command from the control unit 107 or moves the focus lens according to a focus command. The position of the zoom lens and the position of the focus lens are output to the control unit 107.

  A photoelectric conversion unit is two-dimensionally arranged on the light receiving surface of the image sensor 103, and light incident on each photoelectric conversion unit is converted into an electrical signal in accordance with a timing signal output from the control unit 107, and converted. The electrical signal thus output is output to the AFE 104.

  The AFE 104 performs noise removal or amplification of the electrical signal output from the image sensor 103 in accordance with the timing signal output from the control unit 107 and outputs the result to the A / D conversion unit 105.

  The A / D converter 105 converts the electrical signal output from the AFE 104 into a digital signal in accordance with the timing signal output from the controller 107. A digital signal for one screen is temporarily stored in the image buffer 106 as image data of a captured image.

  The control unit 107 operates according to a program stored therein in advance, and controls each unit of the digital camera 101. For example, color interpolation processing, white balance processing, or the like is performed on the image data temporarily stored in the image buffer 106 and displayed on the display unit 108 as a captured image. Alternatively, image compression processing such as JPEG compression processing is performed on the image data after color interpolation processing and white balance processing. The compressed image data in JPEG format is stored in the memory card 109a connected via the memory card I / F 109.

  The display unit 108 displays a menu screen composed of captured images, characters, graphics, and the like according to instructions from the control unit 107. In particular, in this embodiment, a cooking image, a cooking utensil image, a recipe, and the like are displayed.

  The operation unit 110 includes a power button 201, a release button 202, a zoom button 203, a cross key 204 (up key 204a, down key 204b, left key 204c, right key 204d, as shown in the external view of the digital camera 101 in FIG. ), A confirmation button 205 and the like. The user operates the digital camera 101 by operating these operation buttons. The operation information of these operation buttons is output to the control unit 107, and the control unit 107 controls the entire operation of the digital camera 101 according to the operation information input from the operation unit 110. Further, the operation unit 110 selects the item A, the item B, the item C, and the like displayed on the screen of the display unit 108 in FIG. 2 with the cross key 204 and presses the confirm button 205 to confirm the selection. For example, each time the upper key 204a or the lower key 204b of the cross key 204 is pressed, three selection targets of item A, item B, and item C are selected in order, and the selected item is displayed with a thick frame. In addition, items that cannot be displayed on one screen are displayed on another screen, and operations on the next screen and the previous screen are performed, for example, using the left key 204c of the cross key 204 to the previous screen and the right key 204d to the next screen. Can move.

  In the case of FIG. 2, the item B is selected with the cross key 204, and the item B is displayed surrounded by a thick frame indicating that it is selected. When the confirm button 205 is pressed in a state where the items are surrounded by the thick frames in this way, the selection of the items surrounded by the thick frames is confirmed.

  The memory 111 is composed of a nonvolatile storage medium such as a flash memory. The memory 111 stores shooting conditions (such as a shooting mode and a resolution of a shot image) of the digital camera 101. Further, image data after color interpolation processing and white balance processing, JPEG format image data read / written to / from the memory card 109a, and the like are also held in the memory 111 as necessary. In the present embodiment, various databases relating to recipes, foodstuffs, persons, and the like are also stored in the memory 111. The detailed contents of the database will be described in detail later.

The date module 112 is composed of an IC chip having a clock calendar function for obtaining a shooting date and time, and each time an image is taken by the digital camera 101, the date and time at the time of shooting are sent to the control unit 107. The control unit 107 records it in the memory 111 and the memory card 109a together with the captured image.
Database configuration
Here, the configuration of the database stored in the memory 111 will be described in detail. The memory 111 has an area for storing each type of information. The food database (food DB) 111a, the cookware database (cooking tool DB) 111b, the recipe database (recipe DB) 111c, and the person database (person DB) ) 111d and other database (other DB) 111e. Hereinafter, each database will be described in turn.
(Food material DB111a)
The food DB 111a stores food images and food information. The memory 111 may store the food image and the food information corresponding to the food image (food name, characteristics, etc.) as the food DB 111a in advance when the digital camera 101 is shipped, or purchase the digital camera 101. The user may register later. In the present embodiment, when the digital camera 101 is shipped, main food images and food information (food names, features, etc.) corresponding to the food images are stored as the food DB 111a and are not stored as the food DB 111a. The food image and the food information can be additionally registered later.
(Cooking utensil DB111b)
The cooking utensil DB 111b stores cooking utensil images and cooking utensil information. The cooking utensil DB 111b is also registered in advance with the main cooking utensil images and cooking utensil information for discriminating the taken cooking utensils, but in the case of the cooking utensil DB 111b in particular, the cooking utensil held by the user In order to store information such as size and size, the user registers the digital camera 101 after purchase.
(Recipe DB 111c)
The recipe DB 111c stores recipe information such as cooking images, cooking names, and cooking methods. In particular, in the present embodiment, information for evaluating the food arrangement image and the beauty of the arrangement is stored. In the memory 111, recipe information such as main cooking images, cooking names, cooking methods, and information on cooking arrangement is stored in advance as the recipe DB 111c when the digital camera 101 is shipped, but is stored as the recipe DB 111c. For a dish that has not been created, a “recipe registration mode” is provided in the digital camera 101 so that the dish name, dish image, cooking method, etc. of the dish can be additionally registered in the recipe DB 111c.
(Person DB 111d)
The person DB 111d stores face data (facial features), basic information (gender, date of birth, birthplace, etc.), favorite food names and cooking names, weak food names and cooking names, allergy information (allergy information) Person information such as the name of a certain ingredient is stored. For registration in the person DB 111 d, a “person registration mode” is provided in the digital camera 101, and a person image captured by the image sensor 103 and a feature amount of a face are extracted. Then, on the screen for inputting personal information, basic information of the photographed person, favorite foods and dishes, unfavorable foods and dishes, names of allergic foods, etc. are registered in the person DB 111d.
(Other DB111e)
In addition, general information according to sex, age, etc., such as nutrition information required for humans, is stored in the DB 111e. For example, as a nutrition-oriented map by age of “male” and “female”, the degree of importance given to each nutritional item such as calories, fat, protein, sugar, vitamin C, minerals, and dietary fiber is shown for each age. Remember information. The other DB 111e may store the nutrition information as described above at the time of product shipment of the digital camera 101. However, the “other information registration mode” is provided in the digital camera 101 and is similar to the person DB 111d. The user may be allowed to register the digital camera 101 after purchase.
[Configuration of Control Unit 107]
Next, the configuration of the control unit 107 in FIG. 1 will be described in detail. The control unit 107 includes a lens control unit 121, an ingredient recognition unit 122, a recipe creation unit 123, a cooking utensil recognition unit 125, an AF control unit 126, a person recognition unit 127, and a dish evaluation unit 128. The In this embodiment, for easy understanding, the lens control unit 121, the food recognition unit 122, the recipe creation unit 123, the cooking appliance recognition unit 125, the AF control unit 126, the person recognition unit 127, and the dish evaluation unit 128 are: Although depicted as being included in the processing of the control unit 107, the control unit 107 may be configured with a dedicated IC chip or the like. Hereinafter, the processing of each unit will be described in detail.
(Processing of the lens control unit 121 and the AF control unit 126)
The lens control unit 121 moves the position of the zoom lens of the photographing optical system 102 via the lens control unit 121 in accordance with the operation of the zoom button 203 of the operation unit 110. In addition, the AF control unit 126 calculates a defocus amount by a focus detection method such as a hill-climbing method at a predetermined position (AF measurement position) of the image captured in the image buffer 106 to obtain a focus position. The control unit 107 adjusts the position of the focus lens of the photographing optical system 102 via the lens control unit 121 so that the focal position obtained by the AF control unit 126 is obtained.

Here, in the present embodiment, the control unit 107 can input the position of the focus lens of the photographing optical system 102 via the lens control unit 121 and know the distance to the subject. Similarly, the control unit 107 can input the position of the zoom lens of the photographing optical system 102 via the lens control unit 121 and know the zoom magnification (focal length). Then, the control unit 107 combines the distance from the focus lens position to the subject and the zoom magnification obtained from the zoom lens position to determine the size of the photographed subject (size of food, cooking utensils, etc.). ) Can be estimated. Since the technique for estimating the size of the subject based on the distance to the subject and the zoom magnification is well known, detailed description thereof is omitted here.
(Processing of the food material recognition unit 122)
Next, processing of the food material recognition unit 122 will be described. The food material recognition unit 122 compares the captured food image with various food images stored in advance in the food material DB 111 a of the memory 111. Then, the similarity between both images is obtained. Note that the technique for searching for similar images can be realized by utilizing the feature amount of the image, as has already been put to practical use in face recognition technology and the like. The feature amount of an image is a general technique for searching for a similar image by searching for an image having a similar parameter.

For example, since the tomato has a feature that the color is red and the shape is round, the red hue range, the diameter of the round shape, and the like are stored as feature amounts, and if they match this, it is determined that it is a tomato. Similarly, since cucumber has a characteristic that the color is dark green and the shape is long and thin, the hue range of dark green, the ratio of the length and width of the long and thin shape, and the like can be determined as the feature amount. As described above, the recognition accuracy can be increased by using the feature amount of the color or shape. In addition, as described above, the food material recognition unit 122 can also estimate the size of the food material from the subject distance and the focal distance, and thus can also determine the amount of the food material.
(Processing of recipe creation unit 123)
Next, the process of the recipe creation part 123 is demonstrated. The recipe creation unit 123 searches for an appropriate recipe from various dishes stored in the recipe DB 111c according to a photographed person image, food image, cooking utensil owned by the user, or the like. Then, information such as the cooking procedure of the dish selected by the user from the searched dishes and necessary ingredients is displayed on the display unit 108. The recipe creation process will be described in detail later.
(Processing of cookware recognition unit 125)
Next, the process of the cooking utensil recognition unit 125 will be described. The cooking utensil recognition unit 125 performs the process of determining what the cooked cooking utensil is, as in the process for determining the food image described above. Further, in addition to the determination, information indicating that the photographed cooking utensil is a cooking utensil owned by the user is added, and a process of registering in the cooking utensil DB 111b of the memory 111 is performed. Accordingly, the recipe creating unit 123 can know what cooking utensil the user owns, and can provide the user with a cooking method according to the cooking utensil owned by the user.
(Processing of the person recognition unit 127)
Next, processing of the person recognition unit 127 will be described. The person recognizing unit 127 uses the face recognition processing technology as described above, the age (age), gender, facial expression (including face color), number of persons or specific person registered in the person DB 111d. It is possible to perform discrimination processing such as who the person is. Then, according to these pieces of information, what kind of recipe is appropriate is selected and set as one of the conditions when the recipe creation unit 123 creates the recipe.
(Processing of the dish evaluation unit 128)
Next, the process of the dish evaluation unit 128 will be described. The dish evaluation unit 128 evaluates the beauty of the food arrangement (workmanship). As a method for evaluating the beauty of the dish arrangement, a method of extracting features obtained from the arranged food image (sample image) is used. There is a method of comparing the arrangement image stored in the recipe DB 111c as a sample image.
[Method without using sample images]
<Color balance>
The dish evaluation unit 128 checks the color balance of the photographed dish arrangement image. The color balance is determined based on whether or not a predetermined color range such as red, yellow, green, blue, black, and white is included. A point is added each time the number of contained colors increases. Alternatively, not only the dish but also the color of the dish is discriminated. In addition, the color analysis of an image should just discriminate | determine whether it exists in hue ranges, such as red, yellow, and green, for every pixel of a dish or a dish part, for example. In addition, detection of dishes and dishes can be detected by extracting the characteristics of dishes and dishes as in the well-known face recognition technology.

In addition, the number of pixels in a predetermined hue range in the dish arrangement image may be obtained for each hue range, and the ratio of the area in the entire dish image may be obtained. For example, when the dish image is composed of 100 pixels, if the number of pixels in the hue range near red is 30 pixels, the ratio of red is set to 30%. In this way, the ratio of red, yellow, green, blue, black, and white in the cooking arrangement image can be obtained, so that the ratio that looks best for each type of dish can be stored in the recipe DB 111c in advance. For example, it is possible to score the appearance of each dish. For example, in the case of “Chingjaolose”, a Chinese dish, scoring is made so that the green color of pepper is 25%, the reddish brown color of beef is 50%, and the beige color of bamboo shoot is 25%. Alternatively, in the case of “Marvo tofu”, if the white color of tofu is 50%, the reddish brown color of ground pork is 40%, and the green color of green onion is 10%, the score is high. In addition, if there are many portions of dark brown, it may be judged that the portion is burnt, and points may be deducted. As described above, since the ingredients to be used and the colors of the ingredients and the color ratios differ depending on the kind of dish, an optimum evaluation criterion is stored in advance for each kind of dish stored in the recipe DB 111c.
<Shape arrangement>
In addition, the dish evaluation unit 128 checks the shape (including the arrangement) of the dishes that have been photographed. Here, when evaluating the form of serving dishes, it is necessary to evaluate the arrangement of servings when there is a side dish, and the dish dishes etc. in the form of servings. For example, when evaluating dishes to be served on the same plate with a plurality of side dishes, it is said that the dishes should be arranged so that the shape of the arrangement is a triangle. For example, in the case of FIG. 3 (a), each dish is arranged on a plate so as to form a triangle, so in the case of FIG. 3 (b), each dish is arranged in a square on the plate. Because there is, it deducts points. In the case of FIG. 3B, since four types of dishes such as side dishes are arranged, it seems unavoidable to form a square, but by combining the two side dishes in one place, FIG. Since a triangle can be formed as shown in FIG.

  Alternatively, as a method for evaluating in the form of the serving itself, for example, in the case of “chinjaoulose” or “marvo tofu”, it is said that the serving shape should be rounded. For example, in the case of FIG. 4 (a), the dishes are arranged in a circular shape on the plate, so a point is added. In the case of FIG. 4 (b), points are deducted because they are arranged on the plate. .

As described above, the dish evaluation unit 128 can evaluate the dish in the form of the arrangement and the arrangement itself, but the arrangement and the shape that look beautiful according to the type of dish are different, and therefore stored in the recipe DB 111c. It is necessary to previously store different evaluation criteria for each type of dish.
<Disposition amount>
In addition, the dish evaluation unit 128 checks the amount of dishes that have been photographed. Here, when evaluating with the amount of cooking, it can be evaluated by the ratio of the area of the dish to the area of the dish. For example, in the case of a single dish, the area where the dish is arranged is about 60% of the area of the dish, and in the case of about 4 kinds of dishes such as side dishes, the dish is relative to the area of the dish. A high score is about 70% of the area of the arranged portion. In the example of FIG. 5 (a), a dish is placed on the plate at about 60% of the area of the plate, so a point is added. In the example of FIG. 5 (b), the same dish is placed on the plate. Since it is arranged at about 90% of the area, the points will be greatly reduced.

In this way, the dish evaluation unit 128 can evaluate the dish according to the serving amount, but the serving amount that looks beautiful depends on the kind of dish, so that each dish type stored in the recipe DB 111c is different. Different evaluation criteria need to be stored in advance.
<Dispersion of ingredients>
In addition, the dish evaluation unit 128 checks the degree of dispersion of the photographed cooking ingredients. Here, when evaluating by the degree of dispersion of cooking ingredients, each ingredient used in cooking is identified by the color and shape of the ingredients, and the position of each ingredient in the cooking image is dispersed. It can be evaluated by whether or not. For example, in the case of “Chinja Olose” in Chinese cuisine, the green color of pepper, the reddish brown color of beef, and the beige color of bamboo shoots are the main ingredients, so the position of each color block in the cooking image The representative pixel is the position of the material (the coordinate position when the vertical and horizontal pixels are expressed in a two-dimensional coordinate system (x, y)). If a multivariate analysis is performed on a plurality of coordinate positions for each ingredient, a variance value for each ingredient can be obtained. For example, in the case of “Chingjaolose” of Chinese cuisine, the green dispersion value of green pepper: 0.8, the red brown dispersion value of beef: 0.6, and the bamboo beige dispersion value: 0.7. Is set to 0.7. In addition, said method is an example and you may obtain | require the dispersion value of ingredients by another method. Here, in the example of “Chingjaoulose” of Chinese cuisine, the appearance of the arrangement is better when the ingredients are dispersed, so the higher the dispersion value, the higher the score. On the other hand, in the case of “chilled Chinese”, the arrangement is more beautiful when the ingredients such as ham, broiled egg, cucumber, ginger, etc. are placed on the Chinese noodles, so the lower the dispersion value, the higher the score. Become.

In this way, the dish evaluation unit 128 can evaluate the dish according to the dispersion value of the ingredients, but since the dispersion value of the ingredients that look beautiful depends on the type of dish, it is stored in the recipe DB 111c. It is necessary to previously store different evaluation criteria for each type of food.
<Others>
In addition to the above, there are various methods for evaluating the appearance of cooking. For example, there is an evaluation method such that a point is deducted if a dish is arranged flat and a point is added if it is three-dimensional. In this case, by changing the angle at which the food is shot, for example, an image of the dish taken from an empty plate and diagonally, and an image of the dish taken from above, etc. Can be evaluated in three dimensions. Thus, various evaluation methods are conceivable, but any evaluation method may be used in the present embodiment.
[How to use sample images]
Next, a method of using the arrangement image stored in the recipe DB 111c in advance as an arrangement sample image (reference image) will be described.

  The feature amount of the sample image is compared with the feature amount of the photographed arrangement image. As the feature amount, the color balance, the shape of the arrangement, the arrangement amount, the dispersion of the ingredients, etc. described in [Method without using sample image] are used. In [Method without using sample image], it is necessary to store the evaluation criteria for each dish in advance in the recipe DB 111c. In [Method using sample image], it is necessary to store the evaluation criteria in the recipe DB 111c in advance. The evaluation criterion is calculated from the sample image of the arrangement stored in the recipe DB 111c. For example, first, a color feature amount is extracted from a sample image of “chinja ose” stored in the recipe DB 111c. Next, a color feature amount is extracted from the photographed arrangement image. Then, the color feature amount of the sample image and the color feature amount of the photographed image are compared and scored. Here, the number of shades and the dispersion of each color are compared, and if green pepper is included, it should be green, and if cone is included, yellow should be included. In addition, “Chinja Olose” or “Fried Rice” is dispersed sparsely, while “Chilled Chinese” or “Miso Ramen” is hardened and not dispersed. Similarly, the ratio of dishes to dishes can also be evaluated by the ratio between the area of the dishes and the area of the dishes, as described in [Method without Using Sample Images]. For example, first, the ratio of the area of the dish and the area of the dish is obtained from the sample image of “chinja olose” stored in the recipe DB 111c. Next, a ratio between the area of the dish and the area of the dish is obtained from the taken arrangement image. Then, the ratio of the area of the dish of the sample image to the area of the dish is compared with the ratio of the area of the dish of the photographed image to the area of the dish to score. For example, if the ratio of the area of the dish and the area of the dish in the sample image of “Tinjaoulose” is 60% and the ratio of the area of the dish and the area of the photographed image is within the range of 60% ± 5%, a point is added. If it is out of the range of 60% ± 10%, points will be deducted.

  In this manner, the dish evaluation unit 128 evaluates the dish image created by the user by comparing the serving sample image stored in advance in the recipe DB 111c with the photographed dish serving image. Can do.

  In the above description, only one sample image of the arrangement stored in the recipe DB 111c is preliminarily stored. However, a plurality of sample images are stored and scored by comparing with each of the plurality of sample images. You may make it employ | adopt the score of the average score and the highest score as a scoring result. In addition, the user may select a target sample image from a plurality of sample images, and the score for the sample image may be adopted.

  Further, the digital camera 101 may have a communication function so that it can be connected to the Internet or the like, and a dish prepared by downloading a sample image of a dish from a preset server or the like may be evaluated. Furthermore, it is possible to download a dish image created by another person on the Internet instead of the dish created by the user and evaluate it with the digital camera 101. In addition, if the program that performs the dish evaluation process of the digital camera 101 runs on a server on the Internet, various people around the world can evaluate their own dishes and conduct cooking contests. Is possible.

Furthermore, in the above description, feature amounts are extracted and compared from each of the sample image and the photographed image, but the correlation between the two images may be calculated to directly calculate the similarity. Absent. In this case, it is necessary that the size of the dish in the image, the size of the dish, etc. are photographed to approximately the same level.
[Flow of cooking evaluation process]
Next, a series of dish evaluation processes of the dish evaluation unit 128 executed by the digital camera 101 according to the present embodiment will be described with reference to the flowchart of FIG. 6 is processed according to a program stored in advance in the control unit 107. In addition, the shooting mode of the digital camera 101 is set to a state in which the user operates the operation unit 110 and is set to “food evaluation mode”. In this state, as shown in FIG. 7A, the display unit 108 displays a display indicating that it is in the (cooking evaluation mode) and a message “Please shoot food”.

  (Step S251) The user presses the release button 202 of the digital camera 101 to photograph a dish image. The dish image photographed by the image sensor 103 is temporarily stored in the image buffer 106. In this state, as shown in FIG. 7B, the captured dish image is displayed on the display unit 108.

  (Step S <b> 252) The dish evaluation unit 128 of the control unit 107 extracts the feature amount of the dish arrangement image temporarily stored in the image buffer 106. As described above, the feature amount of the dish arrangement image includes color balance, arrangement amount, dispersion of ingredients, and the like.

  (Step S253) The dish evaluation unit 128 of the control unit 107 evaluates the shot dish using the feature amount of the dish image extracted in step S252. For example, according to the evaluation criteria stored for each dish in the recipe DB 111d, such as color balance: 60 points, arrangement amount: 70 points, ingredient dispersion: 50 points, etc. Evaluate features. Alternatively, the evaluation is performed based on the difference in the feature amount from the sample image and the similarity of the image itself as compared with the sample image stored in the recipe DB 111d. During the evaluation process, a message (under evaluation) is displayed on the display unit 108 as shown in FIG.

  (Step S254) The dish evaluation unit 128 of the control unit 107 displays the result evaluated in step S253 on the display unit 108. For example, as shown in FIG. 7 (d), as the (evaluation result), the total score: 80 points and the arrangement of arrangement: 90 points and the color scheme: 70 points are displayed on the display unit 108 as individual evaluation points. When the user confirms the evaluation result, the “evaluation” item is selected with the cross key 204 and the confirmation button 205 is pressed to end the dish evaluation process. At the end of the dish evaluation process, the dish evaluation unit 128 stores the evaluation result as information on the dish in the recipe DB 111c.

  Here, when the user does not like the evaluation result of FIG. 7D, when the “correction” item is selected with the cross key 204 and the confirm button 205 is pressed, the correction mode as shown in FIG. to go into. In the correction mode, when the item to be corrected is selected from the “total: 80 points”, “arrangement: 90 points”, and “color arrangement: 70 points” with the cross key 204 and the confirmation button 205 is pressed, as shown in FIG. A pop-up window will open and you can modify the score. In the case of the same figure, the color scheme: 70 points are being corrected to 90 points. When the correction is completed, when the “end” item is selected with the cross key 204 and the confirmation button 205 is pressed, the screen returns to the state of FIG.

  In FIG. 7 (d), the evaluation results are indicated by points. However, as shown in FIG. 8 (a), the evaluation results may be displayed in a visually easy-to-understand manner using the number of stars such as three stars or five stars. Absent. Further, as shown in FIG. 8B, a comment may be input. In the case of FIG. 8 (b), an example of a screen for inputting the evaluation of seasoning is shown. Since the evaluation of the whole seasoning is 30 points and the sweetness is 3 of 5 grades, the medium is hot and the hotness is 5 of 5 grades. You can see that it was too hot. Alternatively, as shown in FIG. 8C, a comment by text may be input. Moreover, since the evaluation result is preserve | saved as the information of the said dish of recipe DB111c by the dish evaluation part 128, when cooking the same dish next time, the quantity and seasoning of an ingredient are referred with reference to the preserved evaluation result. A modified recipe can be created. The process of changing the recipe contents according to the evaluation result will be described in detail in the next recipe creation process.

[Recipe creation process]
Next, the recipe creation process of the recipe creation unit 122 that creates an optimum recipe using the dish evaluation process described with reference to the flowchart of FIG. 6 will be described in detail with reference to the flowchart of FIG. 9 is processed according to a program stored in advance in the control unit 107. Further, the photographing mode of the digital camera 101 is set to a state where the user operates the operation unit 110 and is set to [recipe creation mode].

  (Step S351) The person recognition unit 127 of the control unit 107 performs a person discrimination process. This step is a subroutine for performing the person discrimination process described above. Therefore, when this subroutine is completed, the photographed person's age, gender, facial expression, number of persons, etc. are known. Alternatively, it is in a state where it is known who is a specific person registered in advance in the person DB 111d.

  (Step S352) For example, as shown in FIG. 10A, the recipe creating unit 123 of the control unit 107 displays a menu screen for selecting “recipe creation” or “food photography” on the display unit 108. Then, the user selects “recipe creation” or “food photography” using the cross key 204 of the operation unit 110 and presses the confirmation button 205. In the case of FIG. 10A, “recipe creation” surrounded by a thick frame is selected. For example, if “foodstuff photography” is selected, the process proceeds to step S353. If “recipe creation” is selected, the process proceeds to step S354.

  (Step S353) The food material recognition unit 122 of the control unit 107 performs a food material discrimination process. This step is a subroutine process for taking a food image and determining what the food is. That is, when this subroutine is finished, it is in a state where it is known what the photographed food is.

  (Step S354) The recipe creation unit 123 displays a dish name list corresponding to the person information determined in step S351 on the display unit 108. At this time, a dish corresponding to the person information determined in the person determination process in step S351 and a dish corresponding to the ingredient information determined in the ingredient determination process in step S353 are retrieved from the recipe DB 111c and a list of dishes is displayed on the display unit 108. To do. For example, when the identified person information is elderly and the photographed food is mackerel, as shown in FIG. 10 (c), for elderly people with low calories such as “mackerel miso boiled” or “mackerel grilled with mackerel” Are displayed on the display unit 108. Alternatively, when the specific person stored in the DB 111e is determined in the person determination process in step S351, the favorite food of the person, the date of birth registered as basic information of the person, and the date module 112 The age is calculated from the input date, and a list of dishes most suitable for the specific person is displayed with reference to a nutrition-oriented map stored in the DB 111e.

  (Step S355) The user selects a dish name for which a recipe is to be created from the dish name list displayed on the display unit. For example, the dish name is selected using the cross key 204 of the operation unit 110 and the confirmation button 205 is pressed.

  In this way, the name of the dish for which a recipe is to be created is determined. And it transfers to the display process of the cooking procedure according to the determined dish name.

  Next, processing after the name of a dish for which a recipe is to be created will be described in the flowchart of FIG. Here, as shown in FIG. 10B, a case where “mackerel miso boiled” surrounded by a thick frame is selected will be described as an example.

  (Step S356) The recipe creation unit 123 displays the ingredients necessary to make the selected dish on the display unit 108. Here, since “mackerel miso boiled” is selected, as shown in FIG. 10 (c), as ingredients and seasonings necessary for making “mackerel miso boiled”, “mackerel: 250 g (or , Half body) ”,“ ginger: 10 g ”,“ miso: 1.5 tablespoon ”,“ sake: 200 cc ”, and the like are displayed on the display unit 108.

  Here, for example, in the dish evaluation process described above, when the evaluation result when the same dish has been made in the past is stored in the recipe DB 111d, the amount of the food material reflecting the evaluation result is displayed. For example, if the past evaluation result is too hot as shown in FIG. 8 (b), the amount of the food with the reduced amount is not the amount of the first registered food (FIG. 10 (c)). (FIG. 10 (c ′)) is displayed. As a result, the amount of miso is reduced from 1.5 tablespoons to 1.0 cup, and it is possible to provide a recipe that suppresses the hotness reflecting the user's past evaluation.

  In this way, the recipe creating unit 123 can display the recipe contents in accordance with the past evaluation results. After confirming the display contents, the user presses the confirmation button 205 and proceeds to the next step.

  (Step S357) The recipe creation unit 123 displays a cooking utensil to be used, for example, as shown in FIG. In the case of FIG. 10 (d), as cooking utensils to be used, "pan (diameter: 20 cm, depth: 8 cm)", "Otama", "Sai chopsticks", "Knife", and "chopping board" , “Stove” and the like are displayed on the display unit 108.

  Here, since information indicating whether or not the cooking utensil possessed by the user is added to the cooking utensil DB 111b, the recipe creating unit 123 determines whether or not the user possesses the cooking utensil to be used. Can be determined. Therefore, in this embodiment, if there is a cooking utensil that the user does not have, another cooking utensil that can be substituted is displayed. For example, if the user does not have a “pan (diameter: 20 cm, depth: 8 cm)” but has a “fry pan (diameter: 30 cm)”, as shown in FIG. “Fry pan (diameter: 30 cm)” is displayed instead of “pan (diameter: 20 cm, depth: 8 cm)”. After confirming the display contents, the user presses the confirmation button 205 and proceeds to the next step.

  (Step S358) The recipe creation unit 123 displays the cooking methods in order, for example, as shown in FIGS. Hereinafter, an example of displaying the cooking method on the display unit 108 will be described.

  As [Procedure 1], as shown in FIG. 11 (a), the description “Place sake in a pan and boil it with high heat” is displayed. When the user confirms the display contents and presses the confirmation button 205, a screen as shown in FIG. 11B is displayed. In the example of FIG. 11 (b), an image in which the sake placed in the pot is boiling is displayed. The user confirms the display contents and presses the confirmation button 205.

  As [Procedure 2], as shown in FIG. 11 (c), an explanation “Lower a mackerel into two pieces and halve it into two parts” is displayed. When the user confirms the display contents and presses the confirmation button 205, a screen as shown in FIG. 11D is displayed. In the example of FIG. 11D, an image of mackerel fillet is displayed. The user confirms the display contents and presses the confirmation button 205.

  As [Procedure 3], as shown in FIG. 12 (a), an explanation “Take 3 tablespoons of sesame in a bowl and melt it with chopsticks” is displayed. When the user confirms the display contents and presses the confirmation button 205, a screen as shown in FIG. 12B is displayed. In the example of FIG. 12B, an image is displayed in which a miso taken in a pot in a pot is melted with chopsticks. The user confirms the display contents and presses the confirmation button 205.

  As [Procedure 4], as shown in FIG. 12 (c), an explanation “Put the mackerel fillet in the pan” is displayed. When the user confirms the display contents and presses the confirmation button 205, a screen as shown in FIG. 12D is displayed. In the example of FIG. 12D, an image of mackerel fillets in a pan is displayed. The user confirms the display contents and presses the confirmation button 205.

  As [Procedure 5], as shown in FIG. 13 (a), an explanation is displayed indicating that the pan is simmered in a high heat for about 5 minutes. When the user confirms the display contents and presses the confirmation button 205, a screen as shown in FIG. 13B is displayed. In the example of FIG. 13 (b), an image in which the pan is boiled with a lid is displayed. The user confirms the display contents and presses the confirmation button 205.

  As [Procedure 6], as shown in FIG. 13 (c), an explanation “Open the lid to reduce the heat and put the miso on the mackerel” is displayed. When the user confirms the display contents and presses the confirmation button 205, a screen as shown in FIG. 13D is displayed. In the example of FIG. 13 (d), an image in which miso is put on mackerel is displayed. The user confirms the display contents and presses the confirmation button 205.

  As [Procedure 7], as shown in FIG. 14 (a), an explanation "Stop the fire when it is boiled and it is finished" is displayed. When the user confirms the display content and presses the confirmation button 205, a screen as shown in FIG. 14B is displayed. In the example of FIG. 14B, an image in which mackerel is boiled in a pan is displayed. The user confirms the display contents and presses the confirmation button 205.

  Finally, as shown in FIG. 14 (c), an example in which mackerel miso boiled is put on a plate is displayed on the display unit 108. In the above description, the cooking image is a still image, but it may be a moving image.

  (Step S359) The dish evaluation unit 128 performs the processing from step S251 to step S254 in the flowchart of FIG. 6 described above. As a result, the evaluation results as shown in FIG. 7D and FIG. 8A are displayed on the display unit 108.

  As described above, the automatic recipe creation apparatus using the digital camera 101 according to the present embodiment evaluates the performance of the photographed dish arrangement image by the process of the dish evaluation unit 128 and displays the evaluation result. It can be effective for improving and double the pleasure of cooking. In particular, since past evaluation results are reflected when the same dish is made next time, a recipe suited to the user's preference can be created.

  In the above-described embodiment, the digital camera 101 is taken as an example of an automatic recipe creation device. However, an already shot dish image, person image, or food material image is stored in the memory card 109a and stored in a personal computer. A memory card 109a may be mounted, and a recipe image may be created by reading out a cooking image, a person image, or a food material image. In this case, the program for performing the dish evaluation process, the person discrimination process, the food ingredient discrimination process, and the cooking utensil discrimination process described in the flowchart of FIG. Here, different from the case of the digital camera 101, instead of taking a cooking image, a person image, a food material image, and a cooking utensil image with the image sensor 103, these images are read out from the memory card 109 a and stored in the memory 111. The stored recipe DB 111c, person DB 111d, other DB 111e, food DB 111a, cooking utensil DB 111b, etc. need to be stored in a storage device such as a hard disk of a personal computer. And if the program which performs the recipe creation process demonstrated with the flowchart of FIG. 9 is run with a personal computer, the recipe according to the cooking image, person image, foodstuff image, and cooking utensil image memorize | stored in the memory card 109a will be shown. Can be created and displayed.

  Thus, the automatic recipe creation apparatus and program according to the present invention can be realized by the digital camera 101 or the personal computer.

1 is a block diagram of a digital camera 101 according to a first embodiment. 1 is an external view of a digital camera 101. FIG. It is explanatory drawing which shows the example of arrangement | positioning of arrangement | positioning. It is explanatory drawing which shows the example of a shape of arrangement | positioning. It is explanatory drawing which shows an example of the amount of arrangement | positioning. It is a flowchart which shows an example of a dish evaluation process. It is explanatory drawing which shows an example of a dish evaluation process. It is explanatory drawing which shows the other example of a display of a dish evaluation result. It is a flowchart which shows a recipe creation process. It is explanatory drawing which shows an example of a recipe creation process. It is explanatory drawing which shows the cooking procedures 1 and 2 of a recipe creation process. It is explanatory drawing which shows the cooking procedures 3 and 4 of a recipe creation process. It is explanatory drawing which shows the cooking procedures 5 and 6 of a recipe creation process. It is explanatory drawing which shows the cooking procedure 7 of a recipe creation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Digital camera 102 ... Shooting optical system 103 ... Imaging element 104 ... AFE
105 ... A / D conversion unit 106 ... image buffer 107 ... control unit 108 ... display unit 109 ... memory card I / F 109a ... memory card 110 ... operation unit 111 ..Memory 112 ... Date module 121 ... Lens control unit 122 ... Food material recognition unit 123 ... Recipe creation unit 125 ... Cookware recognition unit 126 ... AF control unit 127 ... Person Recognizing unit 128 ... Cooking evaluation unit 201 ... Power button 202 ... Release button 203 ... Zoom button 204 ... Cross key 204a ... Up key 204b ... Down key 204c ... Left Key 204d ... Right key 205 ... Confirm button

Claims (9)

An imaging means for taking a dish image,
Image processing means for extracting the feature quantity of the serving from the dish serving image photographed by the imaging means;
A recipe database for storing a cooking image and a cooking standard for each cooking type,
An automatic recipe creating apparatus comprising: an evaluation unit that compares the feature amount of the serving extracted by the image processing unit with the serving standard stored in the recipe database. In the automatic recipe preparation apparatus of Claim 1,
The serving feature amount extracted by the image processing means is determined by at least one of the color scheme of the dish, the arrangement and shape of the dish with respect to the dish, the area ratio of the dish to the dish, and the dispersion of ingredients.
The recipe reference stored in the recipe database is determined by at least one of the color scheme of the dish, the arrangement and shape of the dish relative to the dish, the area ratio of the dish to the dish, and the distribution of ingredients. . In the automatic recipe preparation apparatus of Claim 1 or 2,
Further providing a display means,
The said evaluation means displays an evaluation result on the said display means. The automatic recipe preparation apparatus characterized by the above-mentioned. In the automatic recipe preparation apparatus of Claim 3,
The said evaluation means displays the said evaluation result on the said display means with a numerical value or a symbol. The automatic recipe preparation apparatus characterized by the above-mentioned. In the automatic recipe preparation apparatus as described in any one of Claim 1 to 4,
The said evaluation means memorize | stores the cooking arrangement | sequence image image | photographed with the said imaging means in association with the kind of the said dish of the said recipe database as a past example. The automatic recipe creation apparatus characterized by the above-mentioned. In the automatic recipe preparation apparatus of Claim 5,
The evaluation unit adds and stores the evaluation result of the evaluation unit when storing the dish arrangement image captured by the imaging unit as a past example in association with the type of the dish in the recipe database. An automatic recipe creation device. In the automatic recipe preparation apparatus of Claim 6,
A comment input means for inputting a comment is further provided.
The evaluation means inputs the evaluation result of the evaluation means and the comment input means when storing the dish arrangement image taken by the imaging means as a past example in association with the type of the food in the recipe database. An automatic recipe creation device characterized by adding and storing stored information. In the automatic recipe preparation apparatus of Claim 7,
An automatic recipe creation device, further comprising: a recipe changing means for changing the contents of a recipe stored in the recipe database in accordance with information input by the comment input means. A program processed using a computer,
An image input means for inputting a dish image,
An image processing procedure for extracting the feature amount of the serving from the dish serving image input by the image input means;
And an evaluation procedure for comparing the serving standard stored in the recipe database storing the serving image and the serving standard for each type of dish with the feature value of the serving extracted by the image processing means. Automatic recipe creation program.

Images