JP2023144678A - Information providing apparatus, information providing system, information providing method, and program - Google Patents

Abstract

To provide an information providing apparatus or the like capable of accurately determining dish candidates included in an image.SOLUTION: An information processing apparatus 100 comprises: an acquisition unit 120 for acquiring images and viewing history information that includes recipe information indicating recipes for one or more dishes viewed by a user; an extraction unit 130 for extracting the dish included in the image; a determining unit 140 for determining one or more candidates for the dish included in the image and the priority of each of the one or more candidates, based on the viewing history information; and an output unit 150 for prioritizing and outputting information regarding a candidate with a high priority among the one or more candidates based on the priority of each of the one or more candidates.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to an information providing device, an information providing system, an information providing method, and a program.

Patent Document 1 discloses that a dish name is identified by analyzing the characteristics of an image, the identified dish name is recorded as history information, and when a request from a user is input, the user Disclosed is a device for recommending dishes.

Japanese Patent Application Publication No. 2012-181797

The present disclosure provides an information providing device and the like that can accurately determine food candidates included in an image.

An information providing device according to an aspect of the present disclosure includes an acquisition unit that acquires an image and viewing history information including recipe information indicating recipes for one or more dishes viewed by a user, and extracts dishes included in the image. an extraction unit that determines one or more candidates for the dish included in the image and a priority of each of the one or more candidates based on the viewing history information; and an output unit that outputs information regarding a candidate with a high priority among the one or more candidates based on the priority.

Further, an information providing system according to an aspect of the present disclosure includes the information providing device described above and a terminal, and the terminal includes a photographing unit that generates the image by photographing a dish, and a photographing unit that generates the image by photographing a dish. A communication unit for transmitting data to the information providing device.

Further, an information providing method according to an aspect of the present disclosure acquires an image and browsing history information including recipe information indicating recipes for one or more dishes viewed by a user, and extracts dishes included in the image. , determine one or more candidates for the dish included in the image and a priority of each of the one or more candidates based on the viewing history information, and determine the priority of each of the one or more candidates based on the priority of each of the one or more candidates. Information regarding a candidate with a high priority among one or more candidates is output with priority.

Further, a program according to one aspect of the present disclosure is a program for causing a computer to execute the information providing method.

Note that these general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM. It may be realized by any combination of a circuit, a computer program, and a recording medium.

According to the present disclosure, it is possible to provide an information providing device and the like that can accurately determine food candidates included in an image.

FIG. 1 is a diagram for explaining an overview of an information providing system according to an embodiment. FIG. 2 is a block diagram showing the configuration of the information providing system according to the embodiment. FIG. 3 is a diagram for explaining the processing of the extraction unit according to the embodiment. FIG. 4 is a diagram showing a specific example of candidate information according to the embodiment. FIG. 5 is a flowchart showing the processing procedure of the information providing apparatus according to the embodiment. FIG. 6 is a sequence diagram showing the processing procedure of the information providing system according to the embodiment.

(The circumstances that led to this disclosure)
2. Description of the Related Art Conventionally, there is an apparatus that analyzes the characteristics of an image, that is, performs image analysis to identify a dish included in (pictured in) the image. For example, when an administrator such as a registered dietitian remotely gives advice about a user's diet, the user takes a picture of his or her own meal and sends the image generated by the user to a device used by the registered dietitian. do. Therefore, by using a device used by an administrator to identify a dish included in an image, the device can display the name of the identified dish on a display device such as a display, and the administrator can Easily track what the user has eaten. For example, the administrator considers the user's attributes such as the user's gender and health condition to determine whether the user is eating a healthy diet, and notifies the user of the user's evaluation of the meal. Provide dietary guidance to

A device that performs image analysis as described above is required to accurately identify dishes included in an image through image analysis. However, when specifying a specified dish for which there are other similar dishes with similar characteristics, outputting a candidate dish as a result of image analysis may result in a similar dish being served instead of the specified dish. There is a problem where the wrong dish candidates are output, such as when the wrong dish candidates are output.

In view of such problems, the inventors of the present application have created an information providing device according to the present disclosure.

Below, embodiments of the present disclosure will be described in detail using the drawings. Note that the embodiments described below each represent a specific example of the present disclosure. Therefore, the numerical values, shapes, materials, components, arrangement and connection forms of the components, steps and order of steps, etc. shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims of the present disclosure will be described as arbitrary constituent elements.

Furthermore, each figure is a schematic diagram and is not necessarily strictly illustrated. Moreover, in each figure, the same reference numerals are attached to the same constituent members.

(Embodiment)
[composition]
FIG. 1 is a diagram for explaining an overview of an information providing system 300 according to an embodiment.

The information providing system 300 is a system for identifying the food that the user ate, which is included in the image, by analyzing the image. The information providing system 300 is used, for example, by an administrator who manages users to evaluate the dishes eaten by the users.

The information providing system 300 includes an information providing device 100 and a terminal 200.

The information providing device 100 includes a communication interface for communicating with the terminal 200, a connector to which a notification device 320 and an input device 330 are connected, a processor that controls various processes of the information providing device 100, and a processor that controls various processes of the information providing device 100. The computer is equipped with a memory storing a control program. In this embodiment, the information providing device 100 is a personal computer, and is operated by an administrator via an input device 330 implemented by an arbitrary user interface such as a mouse and/or a keyboard.

The terminal 200 includes a communication interface for communicating with the information providing device 100, the notification device 320, and the input device 330, a processor that controls various processes of the terminal 200, and a memory that stores a control program executed by the processor. It is a computer equipped with the following. In this embodiment, terminal 200 is a smartphone, and is operated by a user by operating a touch panel display included in terminal 200, for example.

For example, the user uses the terminal 200 to take a picture of the food. The terminal 200 transmits an image (image information) generated by photographing the food to the information providing apparatus 100 via the network 310.

The information providing device 100 acquires an image including a user's meal from the terminal 200, determines a candidate dish included in the acquired image, and notifies the administrator of candidate information indicating the candidate dish. Specifically, the information providing device 100 outputs the candidate information to the notification device 320, thereby causing the notification device 320 to notify the candidate information. Furthermore, the information providing device 100 receives an instruction from the administrator indicating a specific candidate among the cooking candidates, and notifies the administrator of cooking information that is information regarding the instructed cooking candidate. Specifically, the information providing device 100 causes the notification device 320 to notify the notification device 320 of the cooking information by outputting the cooking information to the notification device 320.

The notification device 320 is, for example, a display that displays candidate information and cooking information as images. The notification device 320 may be realized by, for example, an amplifier, a speaker, etc. that outputs candidate information and cooking information by voice.

Note that the information providing device 100 and the terminal 200 may each be realized by a personal computer, a tablet terminal, a smartphone, or the like.

FIG. 2 is a block diagram showing the configuration of the information providing system 300 according to the embodiment.

The information providing device 100 includes a communication section 110, an acquisition section 120, an extraction section 130, a determination section 140, an output section 150, and a storage section 160.

Communication unit 110 is a communication interface for communicating with terminal 200. The communication unit 110 may be realized by a connector to which a communication line for wired communication is connected, or may be realized by an antenna, a wireless communication circuit, etc. for wireless communication.

The acquisition unit 120 is a processing unit that acquires images and viewing history information. For example, the acquisition unit 120 acquires images and viewing history information from the terminal 200 via the communication unit 110.

The browsing history information is information including recipe information indicating recipes for one or more dishes that the user has viewed.

Note that the viewing history information may be stored in the storage unit 160. In this case, the acquisition unit 120 may acquire viewing history information from the storage unit 160.

The extraction unit 130 is a processing unit that extracts dishes included in an image. For example, the extraction unit 130 extracts the food (food image) shown in the image by analyzing the image using a technique such as pattern matching.

FIG. 3 is a diagram for explaining the processing of the extraction unit 130 according to the embodiment.

For example, assume that the acquisition unit 120 acquires the image 340. The extraction unit 130 performs image analysis on the image 340 to extract food images included in the image. In the example shown in FIG. 3, the extraction unit 130 extracts a food image M1, a food image M2, a food image M3, a food image M4, a food image M5, a food image M6, and a food image M7 from the image 340. , extract. The extraction unit 130 analyzes each of the extracted food images M1 to M7 to estimate the dishes included in each of the food images M1 to M7. Thereby, the extraction unit 130 extracts the dishes included in the image 340.

The determining unit 140 is a processing unit that determines one or more candidates for dishes included in an image and the priority of each of the one or more candidates, based on viewing history information. For example, the determining unit 140 calculates a probability indicating what type of dish the extracting unit 130 has extracted is likely to be.

For example, the determining unit 140 performs image analysis or the like on the food image M5 extracted by the extracting unit 130 to select fried chicken, fried shrimp, tempura, pork cutlet, croquette, and fried fish as dishes included in the food image M5. decided as a candidate. Further, the determining unit 140 performs an image analysis or the like on the food image M5, and for each of the determined candidates, the probability (possibility) that it is fried chicken is 30%, and the probability that it is fried shrimp is 20%. , the probability is 30% that it is tempura, the probability that it is pork cutlet is 5%, the probability that it is croquette is 14%, and the probability that it is fried fish is 1%.

The determining unit 140 further corrects the calculated accuracy using viewing history information. For example, assume that the user has browsed a recipe for fried chicken, a recipe for fried shrimp, and a recipe for pork cutlet in the past using the Internet or the like. In other words, it is assumed that the viewing history information includes information indicating that the user has viewed a recipe for fried chicken, a recipe for fried shrimp, and a recipe for pork cutlet in the past. In this case, the determining unit 140 determines, for example, that the accuracy of each of fried chicken, fried shrimp, and pork cutlet that has been calculated to be likely to be a dish included in the food image M5 extracted by the extracting unit 130 is relative. Correct each accuracy to improve the target. For example, the determining unit 140 determines the probability of each candidate dish as follows: fried chicken at 35%, fried shrimp at 25%, tempura at 20%, pork cutlet at 10%, croquette at 9%, and fried fish at 1%. Correct it as follows.

Further, the determining unit 140 determines the priorities of the plurality of dish candidates, for example, based on the calculated accuracy. For example, the determining unit 140 determines the priorities of the plurality of dish candidates in descending order of calculated accuracy. For example, if the lower the value, the higher the priority, the determining unit 140 sets the priority of fried chicken to 1, the priority of fried shrimp to 2, the priority of tempura to 3, the priority of pork cutlet to 4, and the priority of croquettes. The priority of each dish candidate is determined, such as 5 for priority 5 and 6 for fried fish.

Thereby, the determining unit 140 generates candidate information indicating the food candidates extracted by the extracting unit 130 and the priorities of the food candidates.

Note that how the determining unit 140 determines (changes) the priority of the food candidates based on the viewing history information may be arbitrarily determined in advance and is not particularly limited. For example, the determining unit 140 corrects to increase the accuracy of the cooking candidates of the recipe viewed by the user by several percentage points, and corrects the accuracy of the cooking candidates of the recipe not viewed by the user to decrease the accuracy by several percentage points. You may. Alternatively, for example, the determining unit 140 may set the food candidates so that the priority of all the food candidates of recipes that have been viewed by the user is higher than the priority of the food candidates of recipes that have not been viewed by the user. The priority may be corrected.

Note that the determining unit 140 determines the priority of each of the one or more candidates based on viewing history information, attribute information indicating attributes of the user, and supplementary information including at least one of status information indicating the user's status. You may decide.

Additional information is information regarding the user. In this embodiment, the supplementary information is information including attribute information and/or status information.

Attribute information includes, for example, the user's age, the user's gender, the user's physique such as height and weight, the user's lifestyle habits such as the amount of daily exercise, the user's health status such as the presence or absence of illness, the region where the user lives, etc. This is information indicating.

Status information is, for example, information indicating the status of the user's behavior, and information indicating whether the user has eaten out (more specifically, information indicating whether the food included in the image was eaten out). ), time zone (more specifically, information indicating the time zone in which the food included in the image was photographed, that is, whether it is breakfast or dinner, etc.).

For example, the acquisition unit 120 acquires additional information from the terminal 200 via the communication unit 110 and stores it in the storage unit 160.

For example, when the user's attribute is male, the determining unit 140 determines the priority of the food candidates so that the probability (priority) of the candidates for fried food such as fried chicken is high. Further, for example, when the user's attribute information indicates that the dish included in the image was generated by being photographed while eating out, the determining unit 140 determines that the dish included in the image was photographed while eating out. In other words, the influence of browsing history information on the priority is lower than when it is not generated by The food candidates are determined so that the amount of increase is small.

For example, as described above, the determining unit 140 corrects the calculated accuracy using viewing history information. For example, assume that a user has previously viewed a recipe for deep-fried chicken using the Internet or the like. In this case, for example, when the image analysis indicates that the probability of fried chicken is 30%, the determination unit 140 corrects the probability of fried chicken to 35% based on the viewing history information. Here, if the incidental information includes information indicating that the dish included in the image was eaten out, the determining unit 140 determines, for example, that the accuracy of fried chicken is 30% based on image analysis. At this time, the accuracy of fried chicken is corrected to 33% based on the viewing history information and additional information.

For example, the acquisition unit 120 acquires supplementary information from the terminal 200 via the communication unit 110. The supplementary information may be stored in the storage unit 160. In this case, the acquisition unit 120 may acquire the supplementary information from the storage unit 160. Further, the supplementary information may be linked to each image. Thereby, for example, the determining unit 140 determines whether or not the dish included in the image was generated by being photographed while eating out based on the accompanying information, and selects the candidate dish based on the determination result. Priority can be determined.

The viewing history information may also include timing information indicating the timing at which each of the one or more cooking recipes was viewed by the user. In this case, the determining unit 140 selects one or more recipes from among the one or more candidates that the user has viewed within a predetermined period from the timing when the image capturing unit 240 generates an image by photographing the dish. The priority of each of one or more candidates may be determined so as to increase the priority of the food candidates created by the food recipe.

The timing information is, for example, information indicating the date and time when the user viewed the recipe. For example, when determining the candidate for the dish extracted by the extraction unit 130, the determining unit 140 determines whether the recipe is viewed at the timing when the dish was photographed (in other words, when the image containing the dish was generated). If the timing is too old, the recipe is not considered in determining the priority of cooking candidates.

Note that the predetermined period may be arbitrarily determined in advance and is not particularly limited. Information indicating the predetermined period is stored in advance in the storage unit 160, for example.

Further, the number of food candidates determined by the determining unit 140 may be one, or two or more. For example, when determining one candidate as a dish, the determining unit 140 may correct the accuracy of the one candidate based on the viewing history information and determine the accuracy as the priority. Alternatively, the determining unit 140 may determine one candidate based on the result of image analysis, and further determine one or more other candidates different from the one candidate based on viewing history information.

The output unit 150 is a processing unit that outputs information regarding a candidate with a high priority among the one or more candidates based on the priority of each of the one or more candidates determined by the determining unit 140. The output unit 150 notifies the administrator of the candidate information, for example, by outputting the candidate information to the notification device 320.

FIG. 4 is a diagram showing a specific example of candidate information according to the embodiment.

As shown in FIG. 4, for example, the notification device 320 displays one or more candidates as candidate information in descending order of priority. The example shown in FIG. 4 shows food candidates that are displayed in order of priority starting from candidate 1.

Note that outputting with priority may mean, for example, displaying a plurality of candidates on the notification device 320 in descending order of priority as shown in FIG. In this case, for example, the notification device 320 may output the candidates by voice in descending order of priority. Alternatively, for example, outputting with priority may mean outputting an arbitrary number of candidates with high priority.

Further, the information regarding the candidates is, for example, information indicating the name of the dish (cooking menu) indicating the candidate. The information regarding the candidate may include information indicating the priority and/or information indicating the nutrients of the candidate.

For example, by operating the input device 330, the administrator selects an appropriate candidate from among the one or more candidates notified by the notification device 320.

Further, for example, the output unit 150 outputs cooking information regarding the selected dish. The output unit 150 notifies the administrator of the cooking information by, for example, outputting the cooking information to the notification device 320.

The cooking information is, for example, nutrient amount information regarding the nutrients of the dish. This allows the administrator to easily grasp the information necessary to provide dietary guidance to the user.

Note that the candidate information may be transmitted to the terminal 200 and notified to the user. For example, the user may select one candidate from one or more candidates indicated by the candidate information by operating the terminal 200. The terminal 200 may transmit information indicating one selected candidate to the information providing apparatus 100.

Further, for example, when the candidate information includes only one candidate, the notification device 320 (or terminal 200) notifies the one candidate and information indicating that the administrator (or user) is to be inquired as to whether or not the candidate is correct. may be done.

Each of the processing units including the acquisition unit 120, the extraction unit 130, the determination unit 140, and the output unit 150 has a memory, a control program stored in the memory, and a CPU (Central Processing Unit) that executes the control program. It is realized from a processor. Further, these processing units may be realized by one memory and one processor, or may be realized by multiple memories and multiple processors, which are different from each other or in an arbitrary combination. Further, these processing units may be realized by, for example, a dedicated electronic circuit.

The storage unit 160 is a storage device that stores information indicating a plurality of cooking menus (cooking names) for the determination unit 140 to determine a candidate dish, and information used by the information providing device 100 such as browsing history information. be. The storage unit 160 is realized by, for example, an HDD (Hard Disk Drive), a flash memory, or the like.

Note that the content of the information stored in the storage unit 160 may be arbitrarily changed, for example, by having the administrator operate the input device 330 or the like.

The terminal 200 includes a communication section 210, a control section 220, a storage section 230, a photographing section 240, an operation section 250, and a display section 260.

The communication unit 210 is a communication interface for communicating with the information providing device 100. In this embodiment, the communication unit 210 is realized by an antenna, a wireless communication circuit, etc. for wireless communication. When the terminal 200 is, for example, a personal computer, the communication unit 210 may be realized by a connector or the like to which a communication line for wired communication is connected.

The control unit 220 is a processing unit that performs various processes executed by the terminal 200. The control unit 220 is realized by, for example, a memory, a control program stored in the memory, and a processor such as a CPU that executes the control program.

The storage unit 230 is a storage device that stores information used by the terminal 200. The storage unit 230 is realized by, for example, an HDD or a flash memory.

The photographing unit 240 is a camera that generates an image including the food by photographing the food. The control unit 220 transmits the image generated by the photographing unit 240 to the information providing apparatus 100 via the communication unit 210.

Note that the control unit 220 may cause the storage unit 230 to store the image in association with, for example, date and time information indicating the date and time when the image was generated. Further, the control unit 220 sends the plurality of images to the communication unit 210 along with user information including, for example, date and time information associated with each image, user name information uniquely indicating the user such as the user's name, and additional information regarding the user. The information may be transmitted to the information providing apparatus 100 via.

The operation unit 250 is a user interface that accepts user operations. For example, when the operation unit 250 receives an input to operate the photographing unit 240, the control unit 220 causes the photographing unit 240 to photograph the food. Further, for example, when the operation unit 250 receives user information, the control unit 220 causes the storage unit 230 to store the user information, or transmits the user information to the information providing device 100 via the communication unit 210. For example, when the user browses cooking recipes using the Internet, the control unit 220 may cause the storage unit 230 to store browsing history information, or provide the browsing history information via the communication unit 210. The information is transmitted to the device 100. In this embodiment, operation section 250 is realized as a touch panel display together with display section 260.

The display unit 260 is a display that displays images. For example, the administrator checks the cooking information and provides dietary guidance to the user. Specifically, for example, the administrator operates the input device 330 to create notification information indicating the quality of the user's meals, points for improvement, etc., and causes the information providing device 100 to send the notification information to the terminal 200. let When the control unit 220 acquires the notification information via the communication unit 210, the control unit 220 causes the display unit 260 to display the notification information.

Note that the operation unit 250 and the display unit 260 may be realized separately like the notification device 320 and the input device 330.

Further, for example, the information providing apparatus 100 may evaluate the food included in the image based on the cooking information and the supplementary information, and may transmit the evaluation result to the terminal 200 as notification information.

[Processing procedure]
Next, processing procedures executed by the information providing apparatus 100 and the information providing system 300 will be described.

<Information provision device>
FIG. 5 is a flowchart showing the processing procedure of the information providing apparatus 100 according to the embodiment.

First, the acquisition unit 120 acquires, for example, an image and viewing history information including recipe information (S110). The acquisition unit 120 acquires images and viewing history information from the terminal 200 via the communication unit 110, for example.

Note that, for example, if images and viewing history information are stored in the storage unit 160, the acquisition unit 120 may acquire the images and viewing history information from the storage unit 160.

Next, the extraction unit 130 extracts the dishes included in the image (S120).

Next, the determining unit 140 determines one or more candidates for the dish extracted by the extracting unit 130 and the priority of each of the one or more candidates based on the viewing history information (S130).

Next, the output unit 150 preferentially outputs information regarding a candidate with a high priority among the one or more candidates determined by the determination unit 140 (S140).

<Information provision system>
FIG. 6 is a sequence diagram showing the processing procedure of the information providing system 300 according to the embodiment. Note that the processing procedure of the information providing apparatus 100 shown in FIG. 6 is explained by adding and/or changing one specific example to the processing procedure of the information providing apparatus 100 shown in FIG. 5.

First, the terminal 200 acquires supplementary information by receiving, for example, a user's input through the operation unit 250 (S210). For example, the control unit 220 causes the storage unit 230 to store additional information received via the operation unit 250.

Next, the terminal 200 obtains viewing history information by receiving, for example, a user's input through the operation unit 250 (S220). For example, the control unit 220 causes the storage unit 230 to store browsing history information received via the operation unit 250.

Next, the terminal 200 is operated by the user to take a picture of the dish, thereby generating an image including the dish (S230). For example, the control unit 220 causes the storage unit 230 to store the image generated by the imaging unit 240.

Next, when the terminal 200 receives an instruction to send information such as an image from the user, the terminal 200 sends the image, viewing history information, and supplementary information to the information providing device 100 (S240).

Note that the terminal 200 may transmit the image, viewing history information, and supplementary information to the information providing apparatus 100 at different timings.

For example, the acquisition unit 120 acquires an image, viewing history information, and supplementary information from the terminal 200 via the communication unit 110 (S111).

Next, the extraction unit 130 extracts the dishes included in the image (S120).

Next, the determining unit 140 determines one or more candidates for the dish extracted by the extracting unit 130 and the priority of each of the one or more candidates, based on the viewing history information and the supplementary information (S131).

Next, the output unit 150 preferentially outputs information regarding a candidate with a high priority among the one or more candidates determined by the determination unit 140 (S140).

Next, for example, the information providing device 100 generates notification information by receiving input from the administrator via the input device 330 (S150).

Next, for example, the information providing device 100 transmits the generated notification information to the terminal 200 (S160).

Next, when the control unit 220 receives notification information from the information providing apparatus 100 via the communication unit 210, the control unit 220 displays the notification information on the display unit 260 (S250).

[Effects etc.]
As described above, the information providing apparatus 100 according to the embodiment includes the acquisition unit 120 that acquires an image and viewing history information including recipe information indicating recipes for one or more dishes viewed by a user, and an extracting unit 130 that extracts the included dishes; a determining unit 140 that determines one or more candidates for the dish included in the image and the priority of each of the one or more candidates based on the viewing history information; The output unit 150 includes an output unit 150 that outputs information regarding a high-priority candidate among the one or more candidates based on the priority of each of the one or more candidates.

According to this, the priority of candidates is determined based not only on the image analysis result but also on the recipe information. Therefore, in addition to the judgment based on the appearance as seen from the image, the user's past actions can be reflected in determining the priority of the food candidates. Therefore, food candidates included in the image can be determined with high accuracy.

Furthermore, for example, the acquisition unit 120 further acquires supplementary information including at least one of attribute information indicating the user's attributes and state information indicating the user's status. Further, for example, the determining unit 140 determines the priority of each of the one or more candidates based on the viewing history information and the attribute information.

According to this, the priority of the candidates is further determined based on the supplementary information. Therefore, in addition to the judgment based on the appearance as seen from the image, additional information such as the user's characteristics and/or personality can be reflected in determining the priority of the food candidates. Therefore, food candidates included in the image can be determined with higher accuracy.

Further, for example, the viewing history information includes timing information indicating the timing at which each of one or more cooking recipes was viewed by the user. For example, the determining unit 140 selects a recipe for one or more dishes from among the one or more candidates that the user views within a predetermined period from the timing when the image capturing unit 240 generates an image by photographing the dish. The priority of each of the one or more candidates is determined so as to increase the priority of the food candidates created using the recipe of the dish.

According to this, by appropriately setting a predetermined period, the priority of a candidate for a dish, such as a recipe for a dish that was viewed very long ago by the user relative to the timing when the dish was photographed, can be prioritized. Recipe information that may be considered to have little influence on decisions can be appropriately excluded. In other words, by using the timing information, the priority can be determined by appropriately selecting recipe information that can be considered to have a large influence on determining the priority of the candidate for the dish. Therefore, food candidates included in the image can be determined with higher accuracy.

Further, the information providing system 300 according to the embodiment includes an information providing device 100 and a terminal 200. A communication unit 210 for transmitting data to the device 100 is provided.

Further, the information providing method according to the embodiment acquires an image and viewing history information including recipe information indicating recipes for one or more dishes viewed by the user (S110), and extracts dishes included in the image. (S120), determines one or more candidates for the dish included in the image and the priority of each of the one or more candidates (S130), and determines the priority of each of the one or more candidates based on the viewing history information. Based on the priority, information regarding a candidate with a high priority among the one or more candidates is output with priority (S140).

Further, the program according to the embodiment is a program for causing a computer to execute the information providing method according to the embodiment.

According to these, effects similar to those of the information providing device 100 described above can be achieved.

(Other embodiments)
Although the information providing apparatus and the like according to the present disclosure have been described above based on the embodiments, the present disclosure is not limited to the above embodiments.

For example, the information providing system 300 may be realized by a plurality of devices, or may be realized by one device. Specifically, in the above embodiment, the information providing system 300 includes the information providing device 100 and the terminal 200, but the information providing system according to the present disclosure includes a single information providing device such as an information terminal. It may be realized as one device or may be realized as a plurality of devices. For example, the information providing system may be implemented as a client server system. When the information providing system is realized by a plurality of devices, the components included in the information providing system described in the above embodiments may be distributed to the plurality of devices in any manner.

Further, for example, in the above embodiments, the processing executed by a specific processing unit may be executed by another processing unit. Further, the order of the plurality of processes may be changed, or the plurality of processes may be executed in parallel.

Further, for example, in the above embodiment, all or part of the components of the processing unit such as the acquisition unit 120 and the extraction unit 130 included in the information providing device 100 may be configured with dedicated hardware, or It may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as an HDD or a semiconductor memory.

Further, for example, the constituent elements of the processing section may be composed of one or more electronic circuits. Each of the one or more electronic circuits may be a general-purpose circuit or a dedicated circuit.

The one or more electronic circuits may include, for example, a semiconductor device, an IC (Integrated Circuit), an LSI (Large Scale Integration), or the like. An IC or LSI may be integrated into one chip or into multiple chips. Here, it is called an IC or LSI, but the name changes depending on the degree of integration, and may be called a system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Furthermore, an FPGA (Field Programmable Gate Array) that is programmed after the LSI is manufactured can also be used for the same purpose.

Additionally, general or specific aspects of the present disclosure may be implemented in a system, apparatus, method, integrated circuit, or computer program product. Alternatively, the computer program may be implemented in a non-transitory computer-readable recording medium such as an optical disk, HDD, or semiconductor memory. Further, the present invention may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

In addition, various modifications that can be thought of by those skilled in the art to this embodiment, and forms constructed by combining components of different embodiments are also within the scope of the present disclosure, as long as they do not depart from the spirit of the present disclosure. included.

The present disclosure can be applied to computers and the like that communicate with terminals.

100 Information providing device 110, 210 Communication unit 120 Acquisition unit 130 Extraction unit 140 Determining unit 150 Output unit 160, 230 Storage unit 200 Terminal 220 Control unit 240 Photographing unit 250 Operation unit 260 Display unit 300 Information providing system 310 Network 320 Notification device 330 Input device 340 Images M1, M2, M3, M4, M5, M6, M7 Cooking images

Claims (6)

an acquisition unit that acquires images and viewing history information including recipe information indicating recipes for one or more dishes viewed by the user;
an extraction unit that extracts dishes included in the image;
a determining unit that determines one or more candidates for dishes included in the image and a priority of each of the one or more candidates, based on the viewing history information;
An information providing apparatus, comprising: an output unit that outputs information regarding a candidate with a high priority among the one or more candidates based on the priority of each of the one or more candidates. The acquisition unit further acquires additional information including at least one of attribute information indicating an attribute of the user and status information indicating a status of the user,
The information providing device according to claim 1, wherein the determining unit determines the priority of each of the one or more candidates based on the viewing history information and the supplementary information. The viewing history information includes timing information indicating when each of the one or more cooking recipes was viewed by the user,
The determining unit selects recipes for the one or more dishes from among the one or more candidates that have been viewed by the user within a predetermined period from the timing when the image capturing unit generated the image by photographing the dish. The information providing device according to claim 1 or 2, wherein the priority of each of the one or more candidates is determined so as to increase the priority of the food candidates created by the food recipe. The information providing device according to any one of claims 1 to 3;
comprising a terminal and
The terminal is
a photographing unit that generates the image by photographing the food;
An information providing system, comprising: a communication unit for transmitting the image to the information providing device. Obtaining an image and browsing history information including recipe information indicating recipes for one or more dishes viewed by the user;
extracting the dishes included in the image;
determining one or more candidates for dishes included in the image and a priority of each of the one or more candidates based on the viewing history information;
An information providing method, wherein information regarding a candidate with a high priority among the one or more candidates is output with priority based on the priority of each of the one or more candidates. A program for a computer to execute the information providing method according to claim 5.

Images