JP2016177637A - Image retrieval device, image retrieval method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately retrieve a dish desired by a user from an image.SOLUTION: A calculation part 12B calculates a first feature quantity of a first image. A determination part 12C retrieves first information and determines dish names respectively corresponding to a plurality of second feature quantities each of whose first similarity to the first feature quantity is not less than a first threshold and categories respectively corresponding to the dish names. A first display control part 12D displays the plurality of dishes name and categories respectively corresponding to the dish names determined in a display part 14A. A reception part 12E receives a selection of at least one dish name and at least one category corresponding to the dish name among the plurality of dish names and categories respectively corresponding to the dish names displayed. A retrieval part 12F retrieves dish names for each of which a second similarity between a category corresponding to a dish name among the dish names included in the first information and the received category corresponding to the received dish name is not less than a second threshold. A second display control part 12G displays the retrieved dish name in the display part 14A.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image search device, an image search method, and a program.

  By analyzing the photographed image, the dish name of the dish included in the photographed image is specified. For example, the name of a dish is specified by analyzing the color and shape of the dish included in the photographed image from the photographed image.

JP 2012-181797 A

  However, many dishes have similar colors and shapes, and it has been difficult to accurately search for desired dishes from images.

  The problem to be solved by the present invention is to provide an image search device, an image search method, and a program capable of accurately searching a user desired dish from an image.

  The image search apparatus according to the embodiment includes an acquisition unit, a calculation unit, a determination unit, a first display control unit, a reception unit, a search unit, and a second display control unit. The acquisition unit acquires a first image including a dish. The calculation unit calculates a first feature amount of the first image. The determination unit classifies the second image including the dish, the second feature amount of the second image, the dish name of the dish included in the second image, and the dish into a plurality of categories according to a predetermined classification rule. Search for first information in which the category is associated in advance, and the dish name corresponding to each of the plurality of second feature amounts having a first similarity with the first feature amount equal to or greater than a first threshold; and The category corresponding to each of the dish names is determined. The first display control unit displays the plurality of determined dish names and the category corresponding to each of the dish names on the display unit. The reception unit receives selection of at least one of the plurality of displayed dish names and the category corresponding to each of the dish names, and at least one of the categories corresponding to the dish name. The search unit has a second similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name of the dish name included in the first information equal to or greater than a second threshold value. Search for the dish name. The second display control unit displays the searched dish name on the display unit.

The block diagram which shows the functional structure of an image search device. The figure which shows an example of the data structure of 1st information. The figure which shows an example of the data structure of 2nd information. The schematic diagram which shows an example of transition of a screen. The schematic diagram which shows an example of a display screen. The flowchart which shows an example of the procedure of an image search process. The figure which shows an example of cooking. The schematic diagram which shows an example of transition of a screen. The schematic diagram which shows an example of a display screen. The schematic diagram which shows an example of transition of a screen. The schematic diagram which shows a search system. The block diagram which shows the hardware structural example.

  Hereinafter, an embodiment of an image search device, an image search method, and a program will be described in detail with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a functional configuration of an image search apparatus 10 according to the present embodiment. The image search device 10 includes a control unit 12, a photographing unit 18, a UI (User Interface) unit 14, and a storage unit 16. The UI unit 14, the storage unit 16, and the photographing unit 18 are connected to the control unit 12 so as to be able to exchange signals. Note that the image search apparatus 10 may be configured not to include the photographing unit 18.

  In the present embodiment, a case where the image search device 10 is a portable terminal that integrally includes a control unit 12, a UI unit 14, a storage unit 16, and a photographing unit 18 will be described. The portable terminal is, for example, a smartphone or a tablet PC (personal computer). Note that the image search device 10 is not limited to a portable terminal. In addition, the image search device 10 may be configured so that at least one of the photographing unit 18, the UI unit 14, and the storage unit 16 and the control unit 12 are separated.

  Hereinafter, the image search apparatus 10 will be described in detail.

  The photographing unit 18 obtains a first image by photographing.

  The first image is an image including food. For example, the first image is a photographed image of an actual dish, a photographed image of a cooking part such as a magazine on which a dish is posted, a photographed image of a dish displayed on a display device, or the like. The first image may be one frame in a moving image composed of a plurality of frames.

  The photographing unit 18 is a known digital camera, digital video camera, or the like. The photographing unit 18 outputs the first image obtained by photographing to the control unit 12.

  The storage unit 16 is a storage medium such as a hard disk drive (HDD) or an internal memory. The storage unit 16 stores first information, second information, and various data.

  FIG. 2 is a diagram illustrating an example of a data structure of the first information. The first information 20 is information in which the second image, the second feature amount, and the category are associated in advance. The first information 20 may be a database, but the form is not limited.

  The second image is an image including food. The second image is the same as the first image. However, the second image is different from the first image in that the second image is stored in the storage unit 16 in advance. The storage unit 16 stores a plurality of second images in advance.

  The second feature amount is a feature amount of the corresponding second image. Specifically, the second feature amount is a numerical value obtained by image analysis of the corresponding second image. More specifically, the second feature amount indicates the feature amount of the dish included in the corresponding second image. That is, the second feature amount is a numerical value obtained by analyzing a cooking area (hereinafter, also referred to as a cooking area) in the corresponding second image.

  For example, the control unit 12 calculates a value obtained by digitizing the outline shape of the cooking area included in the second image as the second feature amount of the second image. That is, the control unit 12 calculates the HoG feature amount of the cooking region included in the second image, the SIFT feature amount of the cooking region included in the second image, or a combination thereof as the second feature amount.

  In addition, the control part 12 is good also considering the numerical value which added the color characteristic of the cooking area | region contained in the 2nd image to the value which digitized the outline shape of the cooking area | region contained in the 2nd image as a 2nd feature-value. . Further, the control unit 12 may calculate only the color feature of the cooking area included in the image as the second feature amount.

  The color characteristics include an RGB color histogram of the cooking area, a YCrCb color histogram of the cooking area, and RGB pixel values of the cooking area, but are not limited thereto.

  Then, the control unit 12 may register the calculated second feature amount in the first information 20 in association with the corresponding second image of the calculation source.

  The dish name indicates the name of the dish included in the corresponding second image. The dish name is, for example, hamburger, curry, shrimp chilli, meatball, but is not limited thereto. For example, the control unit 12 may receive a dish name corresponding to the second image from the UI unit 14 and register it in the first information 20 according to an operation instruction of the UI unit 14 by the user.

  The category indicates each category when the food is classified into a plurality of categories according to a predetermined classification rule. There may be one or more classification rules. The classification rule is a rule for classifying dishes based on the contents of dishes that are difficult to analyze directly from an image. Specifically, the classification rule is, for example, at least one of ingredients included in cooking, nutritional components of cooking, and cooking recipes. The nutritional component indicates, for example, salt, sugar, lipid, protein, calories and the like. The rules may further include ingredients contained in the dish, allergens contained in the dish, and ingredients that should not be eaten due to religion or creed.

  That is, each category when the food is classified into a plurality of categories by the classification rule “food” is a group corresponding to the content of the food. Each category when cooking is classified into a plurality of categories by “calorie” is a group corresponding to the value of calories. Similarly, when a dish is classified into a plurality of categories by “recipe”, each category is a group corresponding to how to make the dish. In addition, when a dish is classified into a plurality of categories by “allergic substances”, each category is a group corresponding to the type of allergic substance contained in the dish. In addition, when a dish is classified into a plurality of categories by “salt” or “sugar”, each category is a group corresponding to the amount of “salt” or “sugar” included in the dish.

  In the present embodiment, each of a plurality of types of classification rules is used to classify dishes into a plurality of categories corresponding to each classification rule.

  In the example shown in FIG. 2, the case where the foodstuff contained in the dish, the calorie of the dish, the recipe of the dish, the allergen contained in the dish, the salt content, and the sugar content is shown as the classification rule. However, the classification rule is not limited to the form shown in FIG.

  In addition, it is preferable that a classification rule contains at least the foodstuff contained in a dish. More preferably, the classification rule includes at least one of the ingredients included in the dish with the corresponding dish name, the nutritional component of the dish with the corresponding dish name, and the recipe with the corresponding dish name.

  In the example shown in FIG. 2, the dish with the dish name “hamburg” is classified into “beef”, “pork”, “onion”, “egg”, “crumbs”, and “milk” when the dishes are classified according to the classification rule “food ingredients”. Belonging to the category. In addition, the dish with the dish name “hamburg” belongs to the category “400 kcal” when the dish is classified by the classification rule “calorie”.

  Next, the second information will be described. FIG. 3 is a diagram illustrating an example of a data structure of the second information 22. The 2nd information 22 is the history information which showed the dish searched as a user desired dish from the 1st picture by processing of control part 12 mentioned below as a history of the dish which the user ate.

  The second information 22 is information in which the user ID, the dish name, the dish image, the time, and the category are associated with each other. The second information may be a database, but the form is not limited. Moreover, the 2nd information 22 should just be the information which shows the log | history which ate a dish, and should just be the information at least including the dish name searched in the control part 12. FIG.

  The user ID is information for identifying the user. The user ID is, for example, identification information of the owner or operator of the image search device 10. The dish name included in the second information 22 is a dish name searched by the processing of the control unit 12 described later. The dish image is a second image corresponding to a dish name retrieved by the process of the control unit 12 described later or the first image used for the retrieval. The time in the second information 22 is the shooting time of the first image used for the search. The time in the second information 22 may include a date and time, or may be only the time. In addition, the time in the second information 22 may be a time when the corresponding dish name is searched by the control unit 12 described later. The category in the second information 22 is a category corresponding to a dish name searched by the control unit 12 described later in the second information 22.

  Returning to FIG. 1, the description will be continued. The UI unit 14 includes a display unit 14A and an input unit 14B. The display unit 14A displays various images. The display unit 14A is a known display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence).

  The input unit 14B is a means for the user to perform various operation inputs. The input unit 14B is, for example, a voice recognition device such as a mouse, a button, a remote control, a keyboard, and a microphone.

  Note that the input unit 14B and the display unit 14A may be configured integrally. Specifically, the input unit 14B and the display unit 14A may be configured as a UI unit 14 having both an input function and a display function. The UI unit 14 includes an LCD with a touch panel.

  The controller 12 is a computer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The control unit 12 controls the entire image search apparatus 10. The control unit 12 may be a circuit other than the CPU.

  The control unit 12 includes an acquisition unit 12A, a calculation unit 12B, a determination unit 12C, a first display control unit 12D, a reception unit 12E, a search unit 12F, a second display control unit 12G, and an update unit 12H. And a communication unit 12I. Some or all of the acquisition unit 12A, the calculation unit 12B, the determination unit 12C, the first display control unit 12D, the reception unit 12E, the search unit 12F, the second display control unit 12G, the update unit 12H, and the communication unit 12I are, for example, , Causing a processing device such as a CPU to execute a program, that is, may be realized by software, may be realized by hardware such as an IC (Integrated Circuit), or may be realized by using software and hardware in combination. May be.

  The acquisition unit 12A acquires the first image. In the present embodiment, the acquisition unit 12A acquires the first image from the imaging unit 18. The acquisition unit 12A may acquire the first image by reading the first image stored in advance in the storage unit 16. Further, the acquiring unit 12A may acquire the first image from the external device via the communication unit 12I. Examples of the external device include a known PC and a WEB server.

  Further, the acquisition unit 12A may acquire the first image by the following method. Specifically, the acquisition unit 12A is first configured to further include functions such as a television tuner that receives broadcast radio waves from a broadcast station (not shown) as content data and a network interface that receives content data from the Internet. .

  The content data is data including a program and metadata indicating the content of the program. Examples of programs include TV (television) broadcast programs, DVD (Digital Versatile Disk) recording media, VOD (Video On Demand) services, etc. Examples include moving images distributed on the World Wide Web), moving images taken with a camera or a mobile phone, video recorders, HDD recorders, DVD recorders, recorded programs recorded on a TV or PC with a recording function, and the like. Metadata is data indicating the contents of a program.

  Then, the control unit 12 displays the program included in the content data on the display unit 14A. Then, in response to an instruction from the user, the input unit 14B instructs to capture an image. That is, the user can input an instruction for capturing an image from the program displayed on the display unit 14A by operating the input unit 14B while checking the program displayed on the display unit 14A.

  Then, in the acquisition unit 12A, when the image capture instruction is received from the input unit 14B, the frame image (sometimes referred to as a frame) displayed on the display unit 14A when the image capture instruction is received. You may acquire as a 1st image. Further, the acquisition unit 12A may capture a frame image before (for example, several seconds before) the frame image displayed on the display unit 14A when receiving the image capture instruction as the first image.

  The calculation unit 12B calculates the first feature amount of the first image acquired by the acquisition unit 12A.

  The first feature amount is a feature amount of the first image. Specifically, the first feature amount is a numerical value obtained by image analysis of the first image. More specifically, the first feature value indicates the feature value of the dish included in the first image. That is, the first feature amount is a numerical value obtained by image analysis of the cooking area in the first image.

  For example, the calculation unit 12B calculates a value obtained by digitizing the outline shape of the cooking area included in the first image as the first feature amount of the first image. That is, the calculation unit 12B calculates the HoG feature amount of the cooking area included in the first image, the SIFT feature amount of the cooking area included in the first image, or a combination thereof as the first feature amount. Note that the calculation unit 12B may use a value obtained by adding the color feature of the cooking area included in the first image to the value obtained by digitizing the contour shape as the first feature amount.

  Note that the calculation unit 12B calculates the first feature value using the same calculation method as the above-described second feature value. For example, it is assumed that the second feature value is a value obtained by digitizing the contour shape of the cooking area included in the second image using the SIFT feature value. In this case, the calculation unit 12B digitizes the contour shape of the cooking area included in the first image using the SIFT feature amount. Then, the calculation unit 12B calculates the numerical value as the first feature amount.

  The determination unit 12C searches the first information 20 and adds a dish name corresponding to each of the plurality of second feature quantities whose first similarity with the first feature quantity is equal to or greater than a first threshold, and each of the dish names. Determine the corresponding category.

  Specifically, the determination unit 12C first receives the first feature amount from the calculation unit 12B. The first feature amount is calculated from the first image by the calculation unit 12B. Then, the determination unit 12C calculates a first similarity that is a similarity between each of the plurality of second feature amounts registered in the first information 20 and the first feature amount calculated by the calculation unit 12B. .

  For example, the degree of similarity is “1” when the first feature quantity and the second feature quantity match, and the degree of similarity is “0.1” when the first feature quantity and the second feature quantity differ by a predetermined value or more. To do. Then, the determination unit 12C sets the first feature value such that the closer the value between the first feature value and the second feature value, the larger (higher) the value from the similarity “0.1” to the similarity “1”. Calculate similarity.

  Specifically, the determination unit 12C performs the Hamming distance between the first feature value and the second feature value, an identification method such as Adaboost, SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), and normalization mutual. The first similarity is calculated using correlation or the like.

  Then, the determination unit 12C searches for a plurality of second feature amounts having a similarity with the first feature amount equal to or more than the first threshold value among the plurality of second feature amounts registered in the first information 20. Then, the determination unit 12 </ b> C reads, from the first information 20, the dish name corresponding to each of the searched plurality of second feature amounts and the category corresponding to each of the plurality of dish names. Then, the determination unit 12C determines a plurality of dish names read from the first information 20 and a category corresponding to each of the plurality of dish names.

  The determination unit 12C may set an arbitrary value in advance as the first threshold value. And the determination part 12C should just memorize | store this 1st threshold value previously. Note that when only one dish name is determined, the determination unit 12C changes the first threshold value to a value smaller than the already set value and repeats the dish name until a plurality of dish names are determined. And the category may be determined. The first threshold value may be changed as appropriate according to an operation instruction of the UI unit 14 by the user.

  The first display control unit 12D displays the plurality of dish names determined by the determination unit 12C and the category corresponding to each of the dish names on the display unit 14A. Note that the category corresponding to the dish name is the category corresponding to the dish name in the first information 20, in other words, the category to which the dish with the dish name belongs.

  The first display control unit 12D displays each of a plurality of dish names and categories corresponding to the dish names on the display unit 14A in a selectable manner. The first display control unit 12D may display each of a plurality of dish names and categories corresponding to each dish name on the display unit 14A so as to be selectable, and the display format is not limited. For example, the first display control unit 12D arranges a plurality of dish names in a tile shape, and sets a category corresponding to each dish in the vicinity of each dish name (for example, the lower part, the upper part, the left part, and the upper part). You may arrange. In addition, the first display control unit 12D arranges a plurality of dish names in parallel from the upper part to the lower part of the display surface of the display part 14A, and near each dish name (for example, the lower part), for each dish. Corresponding categories may be placed.

  The first display control unit 12D may display a plurality of dish names and categories in known display forms such as flip switching, cover switching, ring switching, grouping, and the like. In addition, when one of the plurality of dish names displayed on the display unit 14A is selected by an instruction of the input unit 14B by the user, the first display control unit 12D selects the selected dish name and the dish name. Control may be made so that the category corresponding to is enlarged and displayed.

  In addition, when one of the plurality of dish names displayed on the display unit 14A is selected according to an instruction of the input unit 14B by the user, the first display control unit 12D may select another dish related to the selected dish name. Control may be performed so that the display area of the dish name is highlighted (flashing, changing the color, etc.). Other related dish names are, for example, dish names having a second similarity similar to the selected dish name.

  Note that the first display control unit 12D preferably displays the plurality of dish names and categories determined by the determination unit 12C on the display unit 14A in an arrangement according to the first similarity. For example, it is preferable that the first display control unit 12D displays the dish names in descending order of the first similarity and displays the category corresponding to each dish name in the vicinity of each dish name. In addition, the first display control unit 12D may switch the display order by a predetermined operation (for example, an operation such as long tap, flick, double flick, swipe, double click, etc.) of the user UI unit 14. For example, the first display control unit 12D performs the display order of the dish names and categories on the display unit 14A from the descending order having the first similarity degree in ascending order, from ascending order to descending order, by a predetermined operation of the user UI unit 14. May be switched.

  FIG. 4 is a schematic diagram illustrating an example of a transition of a screen displayed on the UI unit 14 (display unit 14A).

  FIG. 4A is a schematic diagram illustrating an example (first image 30A) of the first image 30 acquired by the acquisition unit 12A. For example, when the acquisition unit 12 </ b> A acquires the first image 30 </ b> A from the imaging unit 18, the user operates the imaging button 32 with the camera portion of the imaging unit 18 facing the target cooking direction. Then, the imaging unit 18 captures the first image 30A, and the acquisition unit 12A acquires the first image 30A.

  FIG. 4B is a schematic diagram illustrating an example (display screen 34A) of a display screen 34 including a plurality of dish names and categories corresponding to each of the dish names displayed by the first display control unit 12D.

For example, the display screen 34 </ b> A includes a dish name image 36 (36 </ b> A to 36 </ b> C) indicating each of a plurality of dish names determined by the determination unit 12 </ b> C and a category image indicating a category corresponding to each dish name of each dish name image 36. _{38 (38A (38A 1 ~38A 3} ), 38B (38B 1 ~38B 3), 38C (38C 1 ~38C 3)) includes a, a search button 40, a decision button 42, a.

  The dish name image 36 (36A to 36C) is a display area for allowing the user to select at least one of the plurality of dish names determined by the determination unit 12C. In the example shown in FIG. 4B, as an example, the display screen 34 has a dish name image 36A indicating the dish name “hamburg”, a dish name image 36B indicating the dish name “curry”, and a dish indicating the dish name “shrimp”. A name image 36C is included.

  The user can select the dish name image 36 (36A to 36C) having a desired dish name by operating the display area of the dish name image 36 (36A to 36C) displayed on the display unit 14A.

Category image _{38 (38A (38A 1 ~38A 3} ), 38B (38B 1 ~38B 3), 38C (38C 1 ~38C 3)) corresponds to each of the plurality of food names determined by determining section 12C, 1 Alternatively, it is a display area for allowing the user to select at least one of a plurality of categories.

In the example shown in FIG. 4B, as an example, the display screen 34A includes a dish name image 36A of the dish name “hamburg”, a dish name image 36B of the dish name “curry”, and a dish name image of the dish name “shrimp”. At the bottom of each display area of 36C, the category image 38 (38A ₁ , 38B ₁ , 38C ₁ ) of the classification rule “food”, the category image 38 (38A ₂ , 38B ₂ , 38C ₂ ) of the classification rule “calorie”, The category image 38 (38A ₃ , 38B ₃ , 38C ₃ ) of the classification rule “recipe” is included as the category image 38 (38A, 38B, 38C).

  The user instructs to operate at least one of the dish name images 36 displayed on the display unit 14A, and further operates at least one display area of the plurality of category images 38 corresponding to the dish name image 36 that has been instructed to operate. By instructing, a desired category 38 corresponding to the dish name of the dish name image 36 that has been instructed to operate can be selected.

That is, when the user selects the category image 38 via the UI unit 14, a category classified according to the classification rule of the selected category image 38 is selected. For example, it is assumed that the user selects the dish name image 36 </ b> A “hamburg” and further selects the category image 38 </ b> A ₁ “food” via the UI unit 14. As a result, the user categorizes the dish name “hamburg” and the classification rule “foodstuff” and belongs to the category of “hamburger” (specifically, “beef” and “pork” which are the contents of the ingredients contained in the hamburger). “Onion” “Egg” “Bread crumb” “Milk”).

  When the UI unit 14 receives a selection by the user, the UI unit 14 outputs the selected dish name and the selected category corresponding to the dish name to the control unit 12.

  The search button 40 is a display area for instructing the user to execute search processing by the search unit 12F. When the user wants to execute search processing by the search unit 12F, after selecting at least one of the plurality of dish name images 36 displayed on the display screen 34 and at least one of the plurality of category images 38, The display area of the search button 40 is instructed via the UI unit 14. Thereby, the UI unit 14 accepts a search instruction.

  The determination button 42 is used to instruct to determine the dish name of the selected dish name image 36 after selecting the desired dish name image 36 that the user wants to determine as the dish name of the dish included in the first image 30. The area to be manipulated. When the determination button 42 is operated by the user, the UI unit 14 receives a determination instruction.

  Note that the first display control unit 12D may display the category itself classified by the classification rule as the category image 38 so that it can be selected. FIG. 5 is a schematic diagram showing an example (display screen 34I) of a display screen 34 including a plurality of dish names and categories corresponding to each of the dish names displayed by the first display control unit 12D.

Display screen 34I shown in FIG. 5, FIG. 4 categories image ₃₈ shown in _{(B) (38A (38A 1} ~38A 3), 38B (38B 1 ~38B 3), 38C (38C 1 ~38C 3)) in place of includes category images _{48 (48A (48A 1 ~48A 6} ), 48B (48B 1 ~48B 5), 48C (48C 1 ~48C 4)). These category images 48 are examples of images indicating categories classified by the classification rule “foodstuff”. In this case, the category of the selected category image 48 is selected by the user selecting the category image 48 via the UI unit 14. For example, it is assumed that the user selects the dish name image 36A “hamburg” and further selects the category image 48A ₁ “beef” via the UI unit 14. As a result, the user has selected the dish name “hamburg” and the category “beef”. When the UI unit 14 receives a selection by the user, the UI unit 14 outputs the selected dish name and the selected category corresponding to the dish name to the control unit 12.

  Returning to FIG. 1, the description will be continued. Here, as described above, the determination unit 12 </ b> C searches the first information 20, and the dish name corresponding to each of the plurality of second feature amounts whose first similarity with the first feature amount is equal to or greater than the first threshold value. And a category corresponding to each of the dish names. The first feature amount and the second feature amount are feature amounts such as contour shapes and colors obtained by image analysis of the images (first image and second image). For this reason, the determination unit 12 </ b> C has a dish name corresponding to the second image of the dish having a similar color and outline shape, although the ingredients and nutritional components included are different from the dish included in the first image. May be determined. For example, meatballs and shrimp chilli differ in categories such as food ingredients and nutrient components, but the shape and color of meatballs and the shape and color of shrimp chilli have a high degree of similarity according to image analysis results. For this reason, when the dish included in the first image is a meatball, the determination unit 12C may determine the dish name “shrimp chilli”.

  Therefore, in the present embodiment, the determination unit 12C determines the category in addition to the dish name. In other words, the determination unit 12C searches the first information 20, searches for a dish name corresponding to each of a plurality of second feature quantities whose first similarity with the first feature quantity is greater than or equal to the first threshold, and the dish name The category corresponding to each is determined. Then, in addition to the determined dish name, the first display control unit 12D displays a category corresponding to each dish name on the display unit 14A.

  Further, the control unit 12 of the image search apparatus 10 according to the present embodiment includes a reception unit 12E, a search unit 12F, and a second display control unit 12G.

  The reception unit 12E includes at least one dish name and the dish name among the plurality of dish names displayed on the UI unit 14 (display part 14A) by the first display control unit 12D and the category corresponding to each of the dish names. Selection of at least one category corresponding to is accepted.

  That is, for example, the user selects at least one of the displayed plurality of dish name images 36 while referring to the display screen 34A shown in FIG. Further, the user selects at least one of the category images 38 corresponding to the selected dish name image 36.

  For example, it is assumed that the dish name of the user's desired dish is not included in the plurality of dish names indicated by the plurality of dish name images 36 displayed on the display unit 14A by the first display control unit 12D. In this case, for example, the user selects a dish name image 36 of a dish similar to the desired dish. Further, the user selects at least one category image 38 of the category similar to the desired dish from the corresponding category images 38 for the selected dish name image 36.

  The category similar to the desired dish is a category that is highly likely to be the same as or similar to the category to which the user-desired dish belongs.

Specifically, it is assumed that the dish desired by the user, that is, the dish included in the first image 30A is the dish having the dish name “Meatball”. In this case, the user selects, for example, the dish name image of the dish name “hamburg” as the dish name of the dish that seems to be closest to the dish name “Meatball” among the plurality of dish names displayed on the UI unit 14. 36A is selected by an operation instruction from the UI unit 14. Further, at least one of the category images 38 (38A _{1 to} 38A ₃ ) corresponding to the selected dish name “hamburg” is selected.

  Then, the UI unit 14 outputs a set of a dish name corresponding to the selected dish name image 36 and a signal indicating the selected category image 38 corresponding to the dish name to the search unit 12F. .

  When a plurality of dish name images 36 are selected by a user operation instruction on the UI unit 14, the UI part 14 selects a dish name corresponding to the selected dish name image 36 and a selection corresponding to the dish name. A plurality of combinations of the signal indicating the category image 38 and the search result are output to the search unit 12F.

  Further, it is assumed that a plurality of category images 38 are selected for one dish name image 36 by an operation instruction of the UI unit 14 by the user. In this case, the UI unit 14 obtains a combination of a dish name corresponding to the selected dish name image 36 and a signal indicating each of the plurality of selected category images 38 corresponding to the dish name, as a search unit 12F. Will be output to.

  Then, the reception unit 12E receives at least one dish from the UI unit 14 (input unit 14B) and a plurality of dish names displayed on the UI unit 14 (display unit 14A) and categories corresponding to the respective dish names. Selection of a name and at least one category corresponding to the dish name is accepted.

  Specifically, for example, the reception unit 12E receives a dish name corresponding to the selected dish name image 36 and a signal indicating the selected category image 38 corresponding to the dish name from the UI unit 14. The accepting unit 12E reads the category corresponding to the accepted dish name and the classification rule represented by the signal indicating the accepted category image 38 in the first information 20. Thereby, the reception unit 12E includes at least one dish name and at least one dish name among the plurality of dish names displayed on the UI unit 14 (display unit 14A) and the category corresponding to each dish name. Accept selection of one category.

  Next, the search unit 12F includes a category corresponding to the dish name among the dish names included in the first information 20 (see FIG. 2), a received category corresponding to the dish name received by the receiving unit 12E, The dish name whose second similarity is equal to or greater than the second threshold is searched.

  The search unit 12F selects the dish name image 36 and the category image 38 corresponding to the dish name image 36 according to an operation instruction from the UI unit 14 by the user, and when the search button 40 is further instructed to operate. In addition, the above search may be performed.

  First, the search unit 12F grasps the combination of the dish name and the category received by the receiving unit 12E. When the reception unit 12E receives a plurality of dish names, the search unit 12F grasps a plurality of combinations of dish names and categories received by the reception unit 12E.

  Then, the search unit 12F searches the first information 20 for the dish name received by the reception unit 12E, and further searches the first information 20 for the category corresponding to the dish name received by the reception unit 12E.

  For example, it is assumed that the combination of the dish name and category received by the receiving unit 12E is a category classified by the dish name “hamburg” and the classification rule “foodstuff”. In this case, the search unit 12F reads the category classified by the classification rule “food” corresponding to the dish name “hamburg” in the first information 20. In the case of the data configuration shown in FIG. 2, the reception unit 12E reads the categories “beef”, “pork”, “onion”, “egg”, “crumbs”, and “milk”.

  Then, the search unit 12F includes the categories “beef”, “pork”, “onion”, “egg”, “bread crumbs”, and “milk” corresponding to the read dish name “hamburg” included in the first information 20 and the first information 20. The similarity with the category classified by the same classification rule “foodstuff” corresponding to each of the other dish names included in 20 is calculated as the second similarity.

  Specifically, the search unit 12F sets the similarity “1” when these categories completely match, and sets the similarity “0.1” when these categories differ by a predetermined value or more. Then, the determination unit 12C sets the second information so that the closer the values of these categories in the first information 20 are, the larger (higher) the value is from the similarity “0.1” to the similarity “1”. Calculate similarity.

  Specifically, the search unit 12F calculates the second similarity using a Hamming distance, an identification method such as Adaboost, SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), normalized cross-correlation, and the like. To do.

  For this reason, for example, the search unit 12F uses a combination of the same categories as the categories “beef”, “pork”, “onion”, “egg”, “crumbs”, and “milk” classified according to the classification rule “food” for the dish name “hamburg”. The higher the second similarity is calculated, the more the dish name is included. Further, the search unit 12F calculates the second similarity having a lower value as the dish name is different from the combination of categories classified by the classification rule “foodstuff” for the dish name “hamburg”.

  It is assumed that the selected category is a category classified by the classification rule “calorie”. In this case, the search unit 12F calculates the higher second similarity as the dish name has a calorie value closer to the category “400 kcal” that is the calorie value for the dish name “hamburg”.

  The search unit 12F calculates the second similarity in the same manner when another classification rule (for example, “calorie” or the like) is selected.

  It is assumed that a plurality of categories are selected for one dish name according to an operation instruction of the UI unit 14 by the user. In this case, the accepting unit 12E accepts selection of a plurality of categories for the dish name. Then, the search unit 12F uses the same classification rule corresponding to each of a plurality of selected categories among the categories corresponding to the selected dish name in the first information 20 and the other dish names in the first information 20. What is necessary is just to calculate the multiplication value of the similarity with each of the classified categories as the second similarity.

  And the search part 12F searches the dish name more than a 2nd threshold value among the 2nd similarity calculated for every dish name contained in the 1st information 20. FIG.

  Therefore, the search unit 12F searches the first information 20 and the first image for the dish name of the user-desired dish using the selected “category” of the “dishes name” selected by the user as a search key. It becomes.

  The second display control unit 12G displays the dish name searched by the search unit 12F on the display unit 14A. Similar to the first display control unit 12D, the category image 38 and the category image 48 (see FIGS. 4 and 5) for selecting a category corresponding to each dish name are used as the dish name image 36 of the corresponding dish name. You may display further in the vicinity.

  The second display control unit 12G displays the dish name searched by the search unit 12F on the display unit 14A so as to be selectable.

  The 2nd display control part 12G should just display the cooking name which the search part 12F searched on the display part 14A so that selection is possible, and the display format is not limited. For example, the second display control unit 12G arranges a plurality of dish names in a tile shape, and category corresponding to each dish name in the vicinity of each dish name (for example, lower, upper, left, upper, etc.). (Category image 38 and category image 48) may be arranged. In addition, the second display control unit 12G arranges a plurality of dish names in parallel from the upper part to the lower part of the display surface of the display part 14A, and each dish name is arranged near each dish name (for example, the lower part). A category (category image 38 or category image 48) corresponding to the above may be arranged.

  The second display control unit 12G may display a plurality of dish names and categories in a known display form such as flip switching, cover switching, ring switching, grouping, and the like. In addition, when one of the dish names displayed on the display unit 14A is selected by an instruction of the input unit 14B by the user, the second display control unit 12G expands the selected dish name and the dish name. You may control to display.

  In addition, when there are a plurality of dish names searched by the search unit 12F, the second display control unit 12G preferably displays the plurality of dish names on the display unit 14A in an arrangement according to the second similarity. For example, it is preferable that the 2nd display control part 12G displays a dish name in descending order from the 2nd high similarity. The second display control unit 12G may switch the display order by a predetermined operation (for example, an operation such as long tap, flick, double flick, swipe, double click, etc.) of the user UI unit 14. For example, the second display control unit 12G performs a predetermined operation of the UI unit 14 of the user from the descending order of the second similarity to the ascending order, conversely from the ascending order to the descending order, and the display order of the dish names on the display unit 14A May be switched.

  For example, it is assumed that the display screen 34 </ b> A illustrated in FIG. 4B is displayed on the UI unit 14 by the first display control unit 12 </ b> D. Then, it is assumed that the dish name searched by the search unit 12F is displayed on the UI unit 14 (display unit 14A) by the reception unit 12E, the search unit 12F, and the second display control unit 12G. In this case, for example, the second display control unit 12G displays a display screen 34B illustrated in FIG.

  FIG. 4C is a schematic diagram illustrating an example (display screen 34B) of the display screen 34 displayed by the second display control unit 12G.

For example, the display screen 34B displays the dish name image 36 (36D to 36F) indicating the dish name searched by the search unit 12F, and the category image 38 (38D (38D (38D) indicating the category corresponding to each dish name of each dish name image 36). 38D _{1 to} 38D ₃ ), 38E (38E _{1 to} 38E ₃ ), 38F (38F _{1 to} 38F ₃ )), a search button 40, and a determination button 42. As described above, the display screen 34B may not include the category image 38.

  The dish name image 36 (36D to 36F) is a display area for allowing the user to select one of the dish names searched by the search unit 12F. In the example shown in FIG. 4C, as an example, the display screen 34B shows the dish name image 36D indicating the dish name “meatball”, the dish name image 36E indicating the dish name “steak”, and the dish name “Hayashi Rice”. The dish name image 36F is included.

  As described above, the dish name displayed on the UI unit 14 (display unit 14A) by the first display control unit 12D is the dish name determined by the determination unit 12C using the first feature amount and the second feature amount. On the other hand, the dish name displayed on the UI part 14 (display part 14A) by the second display control part 12G is searched by using the accepted category corresponding to the dish name accepted by the search part 12F as a search key. The name of the dish.

  Therefore, the dish name displayed on the UI unit 14 by the second display control unit 12G (for example, see FIG. 4C) is the dish name displayed on the UI unit 14 by the first display control unit 12D (for example, FIG. 4). Compared to (B), the probability that the user's desired dish name is included is high.

The category images 38 (38D (38D _{1 to} 38D ₃ ), 38E (38E _{1 to} 38E ₃ ), 38F (38F _{1 to} 38F ₃ )) are the above category images 38 except that the corresponding dish name images 36 are different. _{(38A (38A 1 ~38A 3)} , 38B (38B 1 ~38B 3), 38C (38C 1 ~38C 3)) is the same meaning as. The search button 40 and the decision button 42 are the same as described above.

  When the dish name image 36 of the desired dish name is included in the dish name image 36 displayed on the display unit 14A by the second display control unit 12G, the user selects the dish name image 36 of the desired dish name. . Then, when determining the selected dish name image 36 as a desired dish name of the user, the user instructs the determination button 42 to operate.

  When the user instructs to operate the dish name image 36 and the determination button 42, the UI unit 14 outputs the dish name of the dish name image 36 that has been instructed to operation and a determination signal to the updating unit 12H.

  The update unit 12H receives the dish name and the determination signal of the dish name image 36 from the UI unit 14 through the operation by the user via the display screen 34B displayed on the UI unit 14 by the second display control unit 12G. The dish name of the dish name image 36 is registered in the second information 22 (see FIG. 3). The update unit 12H may accept the dish name by receiving the dish name image 36 from the UI unit 14 and analyzing the dish name indicated by the accepted dish name image 36.

  The updating unit 12H may register the dish name of the accepted dish name image 36 in the second information 22.

  At this time, the updating unit 12H associates the user ID of the user who has performed the operation instruction, the dish name of the accepted dish name image 36, the dish image corresponding to the dish name, the time, and the category. It is preferable to update the second information 22 by registering it in the second information 22.

  The update unit 12H reads the second image corresponding to the received dish name from the first information 20 as a dish image, and registers the second image 22 in the second information 22. Note that the update unit 12H may register the first image 30 acquired by the acquisition unit 12A (that is, the first image 30 that is the calculation source by the calculation unit 12B) in the second information 22 as a cooking image. Further, the update unit 12H may use the shooting time of the dish image or the time (date and time) when the search unit 12F searches for the corresponding dish name as the time to be registered in the second information 22. The updating unit 12H may read the category corresponding to the registered dish name from the first information 20 and register it in the second information 22.

  For this reason, the update part 12H can update the 2nd information 22 used as log | history information.

  The updating unit 12H may transmit the updated second information 22 to an external device such as a server that performs health management.

  Moreover, the update part 12H may update the 1st information 20 (refer FIG. 2) regularly or irregularly. For example, the first information 20 may be updated by periodically accessing an external server in which the dish name, second image, and category are registered. Further, the updating unit 12H may receive the updated information via the communication unit 12I and register it in the first information 20 when the external server is updated. Further, the first information 20 may be changed as appropriate according to an operation instruction of the UI unit 14 by the user. When the information received from the external device or the like does not include the second feature amount or does not include the category classified by the classification rule in the first information 20, the update unit 12H performs the second operation. A feature amount and a category may be generated and registered in the first information 20. The second feature value may be calculated in the same manner as the first feature value. In addition, a category input by the user operating the UI unit 14 may be registered.

  Next, a procedure of image search processing executed by the image search device 10 will be described. FIG. 6 is a flowchart illustrating an example of a procedure of image search processing executed by the image search apparatus 10.

  First, the acquisition unit 12A acquires the first image (step S100). Next, the calculation unit 12B calculates the first feature amount of the first image acquired in step S100 (step S102).

  Next, the determination unit 12C searches the first information 20, and the dish name corresponding to each of the plurality of second feature quantities whose first similarity with the first feature quantity is equal to or greater than the first threshold, and the dish name The category corresponding to each of these is determined (step S104).

  Next, the first display control unit 12D displays the plurality of dish names determined in step S104 and the category corresponding to each of the dish names on the UI unit 14 (display unit 14A) (step S106). By operating the UI unit 14 (input unit 14B), the user operates the UI unit 14 (display unit 14A) to display at least one dish among a plurality of dish names and categories corresponding to each of the dish names. A name and at least one corresponding to the dish name are selected.

  Next, the reception unit 12E includes at least one dish name and at least one dish name among the plurality of dish names displayed on the UI unit 14 (display unit 14A) and the corresponding category. It is determined whether selection of one category has been accepted (step S108). The accepting unit 12E repeats the negative determination until it makes an affirmative determination (determined that the selection has been accepted) in step S108 (step S108: No), and if an affirmative determination is made (step S108: Yes), the process proceeds to step S110.

  Next, the search unit 12F includes the category corresponding to the dish name among the dish names included in the first information 20 (see FIG. 2) and the category received in step S108 corresponding to the dish name received in step S108. The dish name having the second similarity of not less than the second threshold is searched (step S110).

  Next, the second display control unit 12G displays the dish name searched in step S110 on the UI unit 14 (display unit 14A) (step S112).

  Next, the updating unit 12H determines whether or not a desired dish has been selected by the user (step S114). In step S114, the updating unit 12H determines whether or not the dish name image 36 has been received from the UI unit 14 by a user operation via the display screen 34 displayed on the UI unit 14 by the second display control unit 12G. .

  For example, if the update unit 12H determines that the determination signal indicating that the determination is to be made as the dish name of the dish name image 36 and the desired dish name has been received from the UI unit 14, an affirmative determination is made in step S114 (step S114: Yes). On the other hand, for example, if the update unit 12H determines that the search signal has been received from the UI unit 14, the update unit 12H makes a negative determination in step S114 (step S114: No).

  The determination signal may be transmitted by the UI unit 14 to the updating unit 12H when the determination button 42 is instructed by an operation instruction from the user. Further, the search signal may be transmitted from the UI unit 14 to the update unit 12H when the search button 40 is instructed by an operation instruction from the user.

  If the updating unit 12H makes a negative determination in step S114 (step S114: No), the process returns to step S108. On the other hand, when the update unit 12H makes a positive determination in step S114 (step S114: Yes), the process proceeds to step S116.

  In step S116, the updating unit 12H registers the dish name of the dish name image 36 received in step S114 in the second information 22 (see FIG. 3) (step S116). Then, this routine ends.

  As described above, the image search apparatus 10 according to the present embodiment includes the acquisition unit 12A, the calculation unit 12B, the determination unit 12C, the first display control unit 12D, the reception unit 12E, the search unit 12F, A second display control unit 12G. The acquisition unit 12A acquires a first image including a dish. The calculation unit 12B calculates the first feature amount of the first image. The determination unit 12C includes a second image including a dish, a second feature amount of the second image, a dish name of the dish included in the second image, and categories in which the dishes are classified into a plurality of categories according to a predetermined classification rule. The first information 20 that is associated with the first feature amount in advance, the dish name corresponding to each of the plurality of second feature amounts whose first similarity with the first feature amount is equal to or greater than the first threshold, and the name of the dish name The category corresponding to each is determined. The first display control unit 12D displays a plurality of determined dish names and categories corresponding to the respective dish names on the display unit 14A. The accepting unit 12E accepts selection of at least one dish name and at least one category corresponding to the dish name from among the plurality of displayed dish names and categories corresponding to the respective dish names. The search unit 12F selects a dish name having a second similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name that is greater than or equal to the second threshold among the dish names included in the first information 20. Search for. The second display control unit 12G displays the searched dish name on the display unit 14A.

  Here, the dish name displayed on the UI unit 14 (display unit 14A) by the first display control unit 12D is the dish name determined by the determination unit 12C using the first feature amount and the second feature amount. The first feature value and the second feature value are feature values obtained by image analysis, for example, numerical values obtained from colors and contour shapes.

  On the other hand, the dish displayed by the second display control unit 12G on the UI unit 14 (display unit 14A) is searched by the search unit 12F using the received category corresponding to the selected dish as a search key. It is a dish name.

  That is, the dish name (for example, see FIG. 4C) displayed on the UI unit 14 by the second display control unit 12G is searched using a search key that considers not only the appearance of the dish image but also the contents of the dish. It is a thing. Therefore, the dish name displayed on the UI unit 14 by the second display control unit 12G (for example, see FIG. 4C) is the dish name displayed on the UI unit 14 by the first display control unit 12D (for example, FIG. 4). Compared to (B), the probability that the user's desired dish name is included is high.

  FIG. 7 is a diagram illustrating an example of a dish having a dish name displayed by each of the first display control unit 12D and the second display control unit 12G. As shown in FIG. 7A, for example, it is assumed that the dish included in the first image 30 is the dish name “bread”. Here, the dish name displayed on the UI unit 14 by the first display control unit 12D is the dish name determined by the determination unit 12C. For this reason, the first display control unit 12D displays, on the UI unit 14, a dish name (for example, a dish name “Castella” (rectangular cake)) similar in outline shape and color to the dish of the dish name “bread”. There is a case (see the second image 50 in FIG. 7B).

  On the other hand, the dish name displayed on the UI unit 14 by the second display control unit 12G is a dish name searched using a category such as ingredients as a search key. For this reason, the probability that the dish name “bread” included in the first image 30B shown in FIG. 7A is displayed is high.

  Therefore, the image search device 10 according to the present embodiment can search the user-desired dish with high accuracy from the image (first image).

  Note that the control unit 12 analyzes the categories included in the second information 22 to change the second information 22 such as health maintenance and disease prevention of the user identified by the user ID included in the second information 22. Can be used for health care. In addition, as described above, the image search apparatus 10 according to the present embodiment can accurately search for a user-desired dish. Therefore, by analyzing the second information 22 storing the search results, the user's health And information on preferences can be analyzed with high accuracy.

<Second Embodiment>
In the present embodiment, a case will be described in which the search unit searches for a dish name using the first weighting value.

  FIG. 1 is a block diagram showing a functional configuration of an image search apparatus 10A according to the present embodiment. The image search apparatus 10 </ b> A includes a control unit 13 </ b> A, a photographing unit 18, a UI unit 14, and a storage unit 16. The UI unit 14, the storage unit 16, and the photographing unit 18 are connected to the control unit 13A so as to be able to exchange signals. Note that the image search device 10 </ b> A may be configured not to include the photographing unit 18.

  The image search device 10A has the same configuration as that of the first embodiment except that the control unit 12A is provided instead of the control unit 13A.

  The control unit 13A is a computer that includes a CPU, a ROM, a RAM, and the like. The control unit 13A controls the entire image search apparatus 10A. The control unit 13A may be a circuit other than the CPU.

  The control unit 13A includes an acquisition unit 12A, a calculation unit 12B, a determination unit 12C, a first display control unit 122D, a reception unit 122E, a search unit 122F, a second display control unit 122G, and an update unit 12H. And a communication unit 12I. Some or all of the acquisition unit 12A, the calculation unit 12B, the determination unit 12C, the first display control unit 122D, the reception unit 122E, the search unit 122F, the second display control unit 122G, the update unit 12H, and the communication unit 12I , Causing a processing device such as a CPU to execute a program, that is, may be realized by software, may be realized by hardware such as an IC, or may be realized by using software and hardware in combination. .

  The acquisition unit 12A, the calculation unit 12B, the determination unit 12C, the update unit 12H, and the communication unit 12I are the same as those in the first embodiment.

  In the present embodiment, the reception unit 122E includes at least one dish name and the dish among a plurality of dish names displayed on the UI unit 14 by the first display control unit 122D and categories corresponding to the dish names. The selection of at least one category corresponding to the name is accepted. In addition, the reception unit 122E further receives a first weighting value of the category for which selection has been received.

  Therefore, the first display control unit 122D displays the display screen 34 including an image for inputting the first weighting value of each category on the UI unit 14.

  FIG. 8 is a schematic diagram illustrating an example of transition of a screen displayed on the UI unit 14 (display unit 14A) in the present embodiment.

  FIG. 8A is a schematic diagram showing an example (display screen 34C) of a display screen 34 including a plurality of dish names and categories corresponding to each of the dish names displayed by the first display control unit 122D.

  For example, the display screen 34 </ b> C includes the dish name images 36 (36 </ b> A to 36 </ b> C) indicating each of the plurality of dish names determined by the determination unit 12 </ b> C and the first weighting of the category corresponding to each of the dish names of each dish name image 36. A gauge 44 (44A to 44C) for inputting a value, a search button 40, and an enter button 42 are included.

  A button P is displayed on the gauge 44. The button P can move its display position in a predetermined direction (for example, an arrow X direction) on the display screen 34C. The user can input the first weight value of the category of the corresponding dish name image 36 by operating the UI unit 14 to move the position of the button P in the arrow XA direction or the arrow XB direction. For example, a larger first weight value can be input as the position of the button P is moved to the arrow XA side, and a smaller first weight value can be input as the button P is moved to the arrow XB side.

  That is, the user can select the dish name image 36 (36A to 36C) having a desired dish name by operating the display area of the dish name image 36 (36A to 36C) displayed on the display unit 14A. it can. Further, the user operates the display position of the button P displayed on the gauge 44 (44A to 44C) of the category corresponding to each of the dish name images 36 (36A to 36C) to select a category, and The first weight value of the category can be input. In the example illustrated in FIG. 8, the category in which the display position of the button P is operated is the category selected by the user, and the display position after the button P is changed is the first weight value of the category.

  FIG. 8 shows a case where the gauge 44 can collectively set the first weighting value of the category corresponding to the dish name of each dish name image 36.

  The image for setting the first weighting value is not limited to the gauge 44 as shown in FIG. For example, the first display control unit 122D may display an input field for inputting the value of the first weighting value, an instruction button for instructing to make the first weighting larger than the standard value, An instruction button for instructing to reduce the size may be displayed.

  The user's desired cooking name image 36, the category corresponding to the cooking of the cooking name image 36, and the first weighting value are input by the user's operation instruction on the UI unit 14. Then, the reception unit 122E selects at least one dish name and at least one category corresponding to the dish name from among the plurality of dish names displayed on the UI unit 14 and categories corresponding to the dish names. In addition to receiving, the first weighting value of the category for which selection has been received is further received.

  The search unit 122F is set for the category with the similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name included in the first information 20. The first multiplication value multiplied by the first weighting value is defined as the second similarity. Then, the search unit 122F searches for a dish name whose second similarity is greater than or equal to the second threshold.

  Note that the search unit 122F receives the dish name image 36, the category corresponding to the dish name image 36, and the first weighting value according to an operation instruction of the UI unit 14 by the user, and when the search button 40 is further operated. Alternatively, the dish name may be searched from the first information 20.

  First, the search part 122F grasps | ascertains the combination of the dish name and the category which were received by the reception part 122E similarly to the search part 12F of 1st Embodiment. When the reception unit 122E receives a plurality of dish names, the search unit 122F grasps a plurality of combinations of dish names and categories received by the reception unit 122E.

  Then, the search unit 122F searches the first information 20 for the dish name received by the reception unit 122E, and further searches the first information 20 for a category corresponding to the dish name received by the reception unit 122E.

  Then, the search unit 122F reads the received category corresponding to the dish name (for example, dish name “hamburg”) included in the first information 20 (for example, the categories “beef” and “pork” of the classification rule “food ingredients”). ”) And the category of the same classification rule corresponding to each of the other dish names included in the first information 20 (for example, the category of the classification rule“ food ”) is accepted for the category. The first multiplication value obtained by multiplying the first weighting value is calculated as the second similarity. In addition, the category that received the selection corresponding to the dish name that received the selection included in the first information 20 and the category of the same classification rule corresponding to each of the other dish names included in the first information 20 The degree of similarity may be calculated in the same manner as in the first embodiment.

  That is, the search unit 122F calculates the second similarity according to the first weighting value set by the user for the category.

  And the search part 122F searches the dish name more than a 2nd threshold value among the 2nd similarity calculated for every dish name contained in the 1st information 20. FIG.

  For this reason, the search unit 122F uses the category of the dish name selected by the user as a search key, and further takes into account the first information 20 and the first information by adding the first weight value set to the selected category. The dish name of the dish desired by the user is searched from the image.

  The second display control unit 122G displays the dish name searched by the search unit 122F on the display unit 14A. Similar to the first display control unit 122D, the category corresponding to each dish name may be further displayed in the vicinity of the corresponding dish name.

  For example, the second display control unit 122G displays the display screen 34D illustrated in FIG.

  FIG. 8C is a schematic diagram illustrating an example (display screen 34D) of the display screen 34 displayed by the second display control unit 122G.

  For example, the display screen 34D inputs the dish name image 36 (36D to 36F) indicating the dish name searched by the search unit 122F and the first weighting value of the category corresponding to each dish name of each dish name image 36. A gauge 44 (44D to 44F), a search button 40, and a determination button 42 are included.

  In the example shown in FIG. 8C, as an example, the display screen 34 </ b> D shows the dish name image 36 </ b> D indicating the dish name “Meatball”, the dish name image 36 </ b> E indicating the dish name “Steak”, and the dish name “Hayashi Rice”. The dish name image 36F is included.

  As described above, the dish displayed by the second display control unit 122G on the UI unit 14 is a dish searched by the search unit 122F using the received category corresponding to the selected dish as a search key. Name.

  For this reason, the dish name (for example, refer to FIG. 8B) displayed on the UI unit 14 by the second display control unit 122G is the dish name desired by the user as compared to the dish name displayed in FIG. There is a high probability of including.

  The image search process performed by the image search apparatus 10A is performed by the first display control unit 122D in step S106 in FIG. 6, the reception unit 122E in step S108, and the search unit in step S110. The second embodiment is the same as the first embodiment except that the second display control unit 122G performs the process of step S112.

  As described above, in the image search apparatus 10A according to the present embodiment, the reception unit 122E has at least one dish name and the dish among the plurality of displayed dish names and categories corresponding to the dish names. A selection of at least one category corresponding to the name is received, and a first weighting value of the category for which the selection has been received is further received. In addition, the search unit 122F uses the received first weight value as the similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name included in the first information 20. The multiplied first multiplication value is set as the second similarity, and a dish name having the second similarity equal to or higher than the second threshold is searched.

  For this reason, the image search device 10A according to the present embodiment can search for a dish desired by the user from the image (first image) with higher accuracy than in the first embodiment.

<Modification>
In the example illustrated in FIG. 8, the gauge 44 can collectively set the first weighting value of the category corresponding to the dish name of each dish name image 36. That is, in the example illustrated in FIG. 8, the case has been described in which the receiving unit 122E receives the first weight value of the entire category corresponding to the dish name.

  However, the reception unit 122E may individually receive the first weighting value for each of the plurality of categories corresponding to the dish name.

  FIG. 9 is a schematic diagram illustrating an example (display screen 34E) of the display screen 34 including a plurality of dish names and a plurality of categories corresponding to each of the dish names displayed by the first display control unit 122D in the present modification. FIG.

For example, the display screen 34E displays each of a plurality of categories corresponding to each of the dish names of the dish name images 36 (36A to 36C) indicating each of the plurality of dish names determined by the determination unit 12C. gauge ₄₄ for inputting a first weighting value _{(44A (44A 1, 44A 2} , 44A 3), 44B (44B 1, 44B 2, 44B 3), 44C (44C 1, 44C 2, 44C 3)) and , A search button 40 and a decision button 42.

  A button P is displayed on the gauge 44. The button P can move the display position in a predetermined direction (for example, the arrow X direction) on the display screen 34E, as described above. The user operates the UI unit 14 to move the position of the button P in the direction of the arrow XA or the direction of the arrow XB, so that the first weight of a desired category among the plurality of categories corresponding to the corresponding dish name image 36 is obtained. A value can be entered. For example, a larger first weight value can be input as the position of the button P is moved to the arrow XA side, and a smaller first weight value can be input as the button P is moved to the arrow XB side.

That is, the user can use the gauges 44 (44A (44A ₁ , 44A ₂ , 44A ₃ ), 44B (44B ₁ , 44B ₂ , 44B ₃ ), 44C corresponding to each of the dish name images 36 (36A to 36C). By operating the display position of the button P displayed in each of (44C ₁ , 44C ₂ , 44C ₃ )), it is possible to select a category and input a first weight value of the category. In the example illustrated in FIG. 9, the category in which the display position of the button P is operated is the category selected by the user, and the display position after the button P is changed is the first weighting value of the category.

  In response to an operation instruction from the UI unit 14 by the user, a dish name image 36 desired by the user, a category corresponding to the dish name of the dish name image 36, and a first weighting value of the category are input. Then, the reception unit 122E selects at least one dish name and at least one category corresponding to the dish name from among the plurality of dish names displayed on the UI unit 14 and categories corresponding to the dish names. In addition to receiving, the first weighting value of the category for which selection has been received is further received.

  The search unit 122F multiplies the similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name among the dish names included in the first information 20 by the first weighting value. The 1st multiplication value is set as the second similarity, and the dish name having the second similarity equal to or higher than the second threshold is searched. When a plurality of categories are selected for each dish name, each of the accepted plurality of categories corresponding to the accepted dish name is the same as the other dish names included in the first information 20. What is necessary is just to multiply the similarity with each category of a classification rule by the 1st weighting value set to each category (it is set as 1st multiplication value temporarily). The second multiplication value obtained by multiplying the first multiplication value for each category corresponding to each dish name may be set as the second similarity.

  First, the search part 122F grasps | ascertains the combination of the dish name and category which were received by the reception part 122E similarly to the search part 12F of 2nd Embodiment. When the reception unit 122E receives a plurality of dish names, the search unit 122F grasps a plurality of combinations of dish names and categories received by the reception unit 122E.

  Then, the search unit 122F searches the first information 20 for the dish name received by the reception unit 122E, and further receives the received one or more categories corresponding to the dish name received by the reception unit 122E as the first information. Search from 20.

  Then, the search unit 122F includes the received category (for example, the category “beef” of the classification rule “food”) corresponding to the read dish name (for example, the dish name “hamburg”) included in the first information 20. The category of the same classification rule corresponding to each of the other dish names included in the first information 20 (for example, the category “beef” of the classification rule “foodstuff”) is similar to the category “beef”. The first multiplication value obtained by multiplying the received first weighting value is calculated as the second similarity. It should be noted that the similarity between the category corresponding to the selected dish name included in the first information 20 and the category of the same classification rule corresponding to each of the other dish names included in the first information 20 is calculated. May be the same as in the first embodiment.

  In other words, the search unit 122F calculates the second similarity using the first weighting value set by the user for each category.

  For this reason, the image search device 10A according to the present modification can search the user-desired dish from the image (first image) with higher accuracy.

<Third Embodiment>
In the present embodiment, a case will be described in which the search unit further searches for a dish name using the first weighting value and the second weighting value.

  FIG. 1 is a block diagram showing a functional configuration of an image search apparatus 10B according to the present embodiment. The image search device 10 </ b> B includes a control unit 13 </ b> B, a photographing unit 18, a UI unit 14, and a storage unit 16. The UI unit 14, the storage unit 16, and the imaging unit 18 are connected to the control unit 13B so as to be able to exchange signals. Note that the image search device 10B may be configured not to include the photographing unit 18.

  The image search device 10B has the same configuration as that of the first embodiment except that a control unit 13B is provided instead of the control unit 12.

  The control unit 13B is a computer that includes a CPU, a ROM, a RAM, and the like. The control unit 13B controls the entire image search device 10B. The control unit 13B may be a circuit other than the CPU.

  The control unit 13B includes an acquisition unit 12A, a calculation unit 12B, a determination unit 12C, a first display control unit 123D, a reception unit 123E, a search unit 123F, a second display control unit 123G, and an update unit 12H. And a communication unit 12I. A part or all of the acquisition unit 12A, the calculation unit 12B, the determination unit 12C, the first display control unit 123D, the reception unit 123E, the search unit 123F, the second display control unit 123G, the update unit 12H, and the communication unit 12I are, for example, , Causing a processing device such as a CPU to execute a program, that is, may be realized by software, may be realized by hardware such as an IC, or may be realized by using software and hardware in combination. .

  The acquisition unit 12A, the calculation unit 12B, the determination unit 12C, the update unit 12H, and the communication unit 12I are the same as those in the first embodiment.

  In the present embodiment, the reception unit 123E, like the reception unit 122E of the second embodiment, receives a plurality of dish names displayed on the UI unit 14 by the first display control unit 123D and each of the dish names. Among the corresponding categories, at least one dish name and at least one category corresponding to the dish name are selected. In addition, the reception unit 123E further receives the first weight value of the category for which selection has been received. In the present embodiment, the reception unit 123E also includes a second image including a dish having a dish name that has been selected from the plurality of dish names displayed on the UI unit 14 by the first display control unit 123D. A weight value is further received.

  For this reason, the first display control unit 123D has an image for inputting the first weight value of each category, an image for further inputting the second weight value of the second image including the dish of each dish name, Is displayed on the UI unit 14.

  FIG. 10 is a schematic diagram illustrating an example of transition of a screen displayed on the UI unit 14 (display unit 14A) in the present embodiment.

  FIG. 10 is a schematic diagram showing an example (display screen 34G) of a display screen 34 including a plurality of dish names and categories corresponding to each of the dish names displayed by the first display control unit 123D.

  For example, the display screen 34G includes a dish name image 36 (36A to 36C) indicating each of a plurality of dish names determined by the determination unit 12C, a gauge 44 (44A to 44C), a gauge 46 (46A to 46C), A search button 40 and an enter button 42 are included. The gauges 44 (44 </ b> A to 44 </ b> C) are for inputting the first weighting value of the category corresponding to each dish name of each dish name image 36. The gauge 46 (46A to 46C) is for inputting the second weight value of the second image of the dish having the dish name of each dish name image 36.

  The gauge 44 is the same as that of the second embodiment. A button P is displayed on the gauge 46. The button P can move its display position in a predetermined direction (for example, an arrow X direction) on the display screen 34G. The user can input the second weighting value of the second image of the corresponding dish name image 36 by operating the UI unit 14 and moving the position of the button P in the arrow XA direction or the arrow XB direction. ing. For example, a larger second weight value can be input as the position of the button P is moved to the arrow XA side, and a smaller second weight value can be input as the button P is moved to the arrow XB side.

  That is, the user can select the dish name image 36 (36A to 36C) having a desired dish name by operating the display area of the dish name image 36 (36A to 36C) displayed on the display unit 14A. it can. Further, the user operates the display position of the button P displayed on the gauge 44 (44A to 44C) of the category corresponding to each of the dish name images 36 (36A to 36C) to select a category, and The first weight value of the category can be input. In the example illustrated in FIG. 10, the category in which the display position of the button P is operated is the category selected by the user, and the display position after the button P is changed is the first weight value of the category.

  In addition, the user operates the display position of the button P displayed on the gauge 46 (46A to 46C) of the second image corresponding to each of the dish name images 36 (36A to 36C), so that the second image is displayed. The second weight value can be input. In the example shown in FIG. 10, the display position of the gauge 46 after the button P is changed is the second weight value of the second image corresponding to the gauge 46.

  Note that the image for setting the second weighting value is not limited to the gauge 46 shown in FIG.

  According to an operation instruction of the UI unit 14 by the user, the user-desired dish name image 36, the category and the first weight value corresponding to the dish of the dish name image 36, and the second of the second image of the dish of the dish name image 36 A weight value is input. Then, the reception unit 123E selects at least one dish name and at least one category corresponding to the dish name from among the plurality of dish names displayed on the UI unit 14 and the categories corresponding to the dish names. In addition to receiving, the first weighting value of the category for which selection has been received is further received. In addition, the reception unit 123E further receives the second weight value of the second image including the dish having the selected dish name among the displayed dish names.

  The search unit 123F calculates a first multiplication value obtained by multiplying the similarity with the accepted category corresponding to the accepted dish name among the dish names included in the first information 20, by the first weighting value. And the search part 123F calculates the 2nd multiplication value which multiplied the 2nd weighting value to the similarity with the 2nd image containing the dish of the received dish name among the dish names contained in the 1st information 20. FIG. Further, the search unit 123F uses the third multiplication value obtained by multiplying the second multiplication value and the first multiplication value as the second similarity, and searches for a dish name having the second similarity equal to or higher than the second threshold.

  Specifically, first, the search unit 123F calculates the first multiplication value in the same manner as the search unit 122F of the second embodiment.

  In addition, the search unit 123 </ b> F is similar to the second image corresponding to the accepted dish name included in the first information 20 and the second image corresponding to each of the other dish names included in the first information 20. Each time, a second multiplication value obtained by multiplying the second weight value received for the second image is calculated.

  Then, the search unit 123F calculates a third multiplication value obtained by multiplying the first multiplication value and the second multiplication value as the second similarity.

  That is, the search unit 123F calculates the second similarity according to both the first weighting value set by the user for the category and the second weighting value set by the user for the second image.

  And the search part 123F searches the dish name more than a 2nd threshold value among the 2nd similarity calculated for every dish name contained in the 1st information 20. FIG.

  For this reason, the search unit 123F uses the category of the dish name selected by the user as a search key, and further sets the first weighting value set for the selected category and the second image of the selected dish name. By adding the second weighted value, the dish name of the dish desired by the user is searched from the first information 20 and the first image.

  The second display control unit 123G displays the dish name searched by the search unit 123F on the display unit 14A. Similar to the first display control unit 123D, the category corresponding to each dish name may be further displayed in the vicinity of the corresponding dish name.

  The image search process performed by the image search apparatus 10B is performed by the first display control unit 123D performing the process of step S106 in FIG. 6, the reception unit 123E performing the process of step S108, and the process of step S110. The second embodiment is the same as the first embodiment except that the second display control unit 123G performs the process of step S112.

  As described above, in the image search device 10B according to the present embodiment, the reception unit 123E has at least one dish name and the dish among the displayed plurality of dish names and categories corresponding to the respective dish names. A selection of at least one category corresponding to the name is received, and a first weighting value of the category for which the selection has been received is further received. In addition, the reception unit 123E further receives the second weight value of the second image including the dish having the selected dish name among the displayed dish names.

  Then, the search unit 123F performs second weighting on the similarity between the second image corresponding to the dish name and the second image including the dish having the accepted dish name among the dish names included in the first information 20. A third multiplication value obtained by multiplying the second multiplication value obtained by multiplying the value by the first multiplication value is defined as the second similarity. Then, the search unit 123F searches for a dish name having the second similarity equal to or higher than the second threshold.

  For this reason, compared with the first embodiment, the user-desired dish can be searched from the image (first image) with higher accuracy.

<Fourth embodiment>
In the first to third embodiments, the case where the storage unit 16 is provided in the image search devices 10, 10A, and 10B has been described. In the present embodiment, a case will be described in which the storage unit 16 is provided in a storage device connected to the image search devices 10, 10A, and 10B via a communication line.

  FIG. 11 is a schematic diagram showing the search system 400. In the search system 400, an image search device 410 and a storage device 412 are connected via a communication line 414.

  The image search device 410 includes control units 12, 13 </ b> A, 13 </ b> B, a UI unit 14, and a photographing unit 18. The control units 12, 13A, 13B, the UI unit 14, and the imaging unit 18 are the same as the image search devices 10, 10A, 10B of the first to third embodiments. That is, the image search device 410 has the same configuration as the image search devices 10, 10A, and 10B of the first to third embodiments except that the storage unit 16 is not provided.

  In addition, the same code | symbol is provided to the same function part as 1st Embodiment-3rd Embodiment, and detailed description is abbreviate | omitted.

  The communication line 414 includes a wired communication line and a wireless communication line. The storage device 412 is a device that includes the storage unit 16 and includes a known PC, various servers, and the like. The storage unit 16 is the same as that in the first embodiment.

  As illustrated in FIG. 11, the storage unit 16 is configured separately from the image search device 410 and is provided in the storage device 412 connected via the communication line 414. As a result, the same storage unit 16 can be accessed from a plurality of image search devices 410, and centralized management of data stored in the storage unit 16 becomes possible.

  Next, the hardware configuration of the image search apparatuses 10, 10A, 10B, and 410 according to the first to fourth embodiments will be described. FIG. 12 is a block diagram illustrating a hardware configuration example of the image search apparatuses 10, 10 </ b> A, 10 </ b> B, 410, and 412 according to the first to fourth embodiments.

  The image search apparatuses 10, 10 </ b> A, 10 </ b> B, and 410 according to the first to fourth embodiments include a communication I / F unit 308, a display unit 310, an input unit 312, a CPU 300, a ROM 302, a RAM 304, and an HDD 306. Are connected to each other by a bus 314 and have a hardware configuration using a normal computer.

  The CPU 300 is an arithmetic device that controls the overall processing of the image search devices 10, 10A, 10B, and 410 according to the first to fourth embodiments. The RAM 304 stores data necessary for various processes performed by the CPU 300. The ROM 302 stores programs and the like that implement various processes performed by the CPU 300. The HDD 306 stores the data stored in the storage unit 16 described above. The communication I / F unit 308 is an interface for connecting to an external device or an external terminal via a communication line or the like and transmitting / receiving data to / from the connected external device or external terminal. The display unit 310 corresponds to the display unit 14A described above. The input unit 312 corresponds to the input unit 14B described above.

  A program for executing search processing executed by the image search apparatuses 10, 10A, 10B, and 410 of the first to fourth embodiments is provided by being incorporated in advance in the ROM 302 or the like.

  The program for executing the search processing executed by the image search apparatuses 10, 10A, 10B, and 410 according to the first to fourth embodiments can be installed in these apparatuses or executed. The file may be recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

  In addition, a program for executing search processing executed by the image search apparatuses 10, 10A, 10B, and 410 according to the first to fourth embodiments is executed on a computer connected to a network such as the Internet. And may be provided by being downloaded via a network. Also, a program for executing search processing executed by the image search apparatuses 10, 10A, 10B, and 410 according to the first to fourth embodiments is provided or distributed via a network such as the Internet. You may comprise as follows.

  A program for executing search processing executed by the image search apparatuses 10, 10A, 10B, and 410 according to the first to fourth embodiments has a module configuration including the above-described units. As actual hardware, the CPU 300 reads out and executes a program for executing the search process from a storage medium such as the ROM 302, and the respective units are loaded onto the main storage device, and the respective units are loaded onto the main storage device. It is to be generated.

  Although several embodiments and modifications of the present invention have been described above, these embodiments and modifications are presented as examples and are intended to limit the scope of the invention. Not done. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

10, 10A, 10B, 410 Image search device 12A Acquisition unit 12B Calculation unit 12C Determination units 12D, 122D, 123D First display control units 12E, 122E, 123E Reception units 12F, 122F, 123F Search units 12G, 122G, 123G Display control unit

Claims (9)

An acquisition unit for acquiring a first image including dishes;
A calculation unit for calculating a first feature amount of the first image;
A second image including food, a second feature amount of the second image, a food name of the food included in the second image, and the category in which the food is classified into a plurality of categories according to a predetermined classification rule; Are searched for in advance, and the dish name corresponding to each of the plurality of second feature quantities having a first similarity with the first feature quantity equal to or greater than a first threshold, and the name of the dish name A determination unit for determining the category corresponding to each;
A first display control unit configured to display on the display unit the plurality of determined dish names and the category corresponding to each of the dish names;
A reception unit that accepts selection of at least one of the plurality of displayed dish names and the category corresponding to each of the dish names and at least one of the categories corresponding to the dish name;
Among the dish names included in the first information, the dish name having a second similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name is a second threshold or more. A search part to search;
A second display control unit for displaying the searched dish name on the display unit;
An image search apparatus comprising: The reception unit
The selection of at least one of the plurality of displayed dish names and the category corresponding to each of the dish names and the selection of at least one of the categories corresponding to the dish names are accepted. Further accepting a first weighting value for said category;
The search unit
A first multiplication in which the similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name among the dish names included in the first information is multiplied by the first weight value. The value is the second similarity, and the dish name having the second similarity equal to or higher than the second threshold is searched.
The image search device according to claim 1. The reception unit
Further receiving a second weighting value of the second image including a dish having the selected dish name among the plurality of displayed dish names;
The search unit
Of the dish names included in the first information, the second weighting value is multiplied by the similarity between the second image corresponding to the dish name and the second image including the dish of the accepted dish name. Searching for the dish name having the second similarity value equal to or higher than the second threshold, with the third multiplication value obtained by multiplying the second multiplication value and the first multiplication value as the second similarity.
The image search device according to claim 2.   The image search device according to claim 1, wherein the classification rule includes a food material included in a dish.   The image search device according to claim 1, wherein the classification rule includes at least one of a food material included in a dish, a nutritional component of the dish, and a corresponding recipe of the dish.   The said 1st display control part is the arrangement | positioning according to the said 1st similarity degree, and displays the said category corresponding to each of the determined several said dish name and this dish name on the said display part. Image search device.   The image search device according to claim 1, wherein the second display control unit displays the searched dish name on the display unit in an arrangement according to the second similarity. Obtaining a first image including food;
Calculating a first feature amount of the first image;
A second image including food, a second feature amount of the second image, a food name of the food included in the second image, and the category in which the food is classified into a plurality of categories according to a predetermined classification rule; Are searched for in advance, and the dish name corresponding to each of the plurality of second feature quantities having a first similarity with the first feature quantity equal to or greater than a first threshold, and the name of the dish name Determining the category corresponding to each;
Displaying a plurality of determined dish names and the category corresponding to each of the dish names on a display unit;
Receiving a selection of at least one of the plurality of displayed dish names and the category corresponding to each of the dish names, and at least one of the categories corresponding to the dish name;
Among the dish names included in the first information, the dish name having a second similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name is a second threshold or more. Searching, and
Displaying the searched dish name on the display unit;
Image search method including Obtaining a first image including food;
Calculating a first feature amount of the first image;
A second image including food, a second feature amount of the second image, a food name of the food included in the second image, and the category in which the food is classified into a plurality of categories according to a predetermined classification rule; Are searched for in advance, and the dish name corresponding to each of the plurality of second feature quantities having a first similarity with the first feature quantity equal to or greater than a first threshold, and the name of the dish name Determining the category corresponding to each;
Displaying a plurality of determined dish names and the category corresponding to each of the dish names on a display unit;
Receiving a selection of at least one of the plurality of displayed dish names and the category corresponding to each of the dish names, and at least one of the categories corresponding to the dish name;
Among the dish names included in the first information, the dish name having a second similarity between the category corresponding to the dish name and the accepted category corresponding to the accepted dish name is a second threshold or more. Searching, and
Displaying the searched dish name on the display unit;
A program that causes a computer to execute.

Images