JP2021086215A - Palatability estimation system, control method thereof, and program - Google Patents

Abstract

To provide an accurate food palatability estimation system.SOLUTION: A palatability estimation system for estimating palatability of food comprises: acquisition means which acquires data including image data of a food, palatability evaluation information of the food, and user information of a user who has evaluated the food; determination means which determines a weight for the data on the basis of the user information included in the data; generation means which generates one piece or a plurality of pieces of image data for learning from the image data included in the data in accordance with the weight determined by the determination means; and learning means which generates a learned model for estimating palatability of the food by performing learning processing with the one piece or a plurality of pieces of image data for learning generated by the generation means as input data and the evaluation information included in the data as teacher data.SELECTED DRAWING: Figure 4

Description

The present invention relates to a deliciousness estimation system for estimating the deliciousness of food, a control method thereof, and a program.

There are many consumer demands to know the deliciousness of agricultural products such as vegetables and fruits before purchasing and to purchase as delicious as possible. However, in general, it is not possible to process food before purchase and analyze its deliciousness. To solve this problem, a system for estimating the deliciousness by AI (Artificial Intelligence) from the appearance information of foods and the like can be considered. In the AI system for estimating the deliciousness judged from the taste that appeals to human senses, many people collect as much learning data as possible and learn as much as possible in order to obtain a convincing evaluation. Need to repeat. However, in a trained model created simply from a large amount of training data, the estimation accuracy in AI does not increase. It is necessary to collect as much accurate learning data as possible.

For example, in Patent Document 1, among many learning data, learning data satisfying a predetermined standard is defined as expert data, and other data is defined as non-expert data. A technique has been proposed in which highly reliable data is increased and a highly accurate classification model is constructed by adding reliability compared to expert data to non-expert data.

Japanese Unexamined Patent Publication No. 2009-110064

In Patent Document 1, a classification model is constructed even when a small amount of high-quality learning data is used by adopting a part of the data that does not meet the predetermined criteria as the learning data. However, the accuracy of the learning model is reduced because the data that does not meet the predetermined criteria is adopted as the learning data.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a highly accurate taste estimation system.

In order to solve the above problems, the present invention has the following configuration. That is, it is a deliciousness estimation system that estimates the deliciousness of food, and is an acquisition means for acquiring data including image data of the food, evaluation information of the deliciousness of the food, and user information of the user who evaluated the food. , A determination means for determining a weight for the data based on the user information included in the data, and one or a plurality of learning images from the image data included in the data according to the weight determined by the determination means. A generation means for generating data and one or a plurality of image data for learning generated by the generation means are used as input data, and evaluation information included in the data is used as teacher data for learning processing to obtain food. It is provided with a learning means for generating a trained model for estimating the taste.

According to the present invention, it is possible to provide a highly accurate taste estimation system.

The figure which shows the example of the whole structure of the deliciousness estimation system which concerns on 1st Embodiment. The figure which shows the example of the hardware configuration of the deliciousness estimation system which concerns on 1st Embodiment. The figure which shows the example of the software structure of the deliciousness estimation system which concerns on 1st Embodiment. The figure for demonstrating the operation outline of the deliciousness estimation system which concerns on 1st Embodiment. The flowchart of the learning phase of the deliciousness estimation system which concerns on 1st Embodiment. The flowchart of the learning phase of the deliciousness estimation system which concerns on 1st Embodiment. The flowchart of the estimation phase of the deliciousness estimation system which concerns on 1st Embodiment. The conceptual diagram which shows the input / output of the learning model which concerns on 1st Embodiment. The figure which shows the configuration example of the UI of the deliciousness estimation system which concerns on 1st Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

<First Embodiment>
[System configuration]
FIG. 1 is a diagram showing an example of the overall configuration of a deliciousness estimation system according to an embodiment of the present invention. In FIG. 1, the client terminal 102, the food management server 103, the data collection server 104, and the estimation server 105 are communicably connected via the Internet 100 and the local network 101. The Internet 100 and the local network 101 are so-called communication networks realized by, for example, LAN, WAN, telephone lines, dedicated digital lines, ATMs, frame relay lines, cable TV lines, wireless lines for data broadcasting, and the like. Each network is not particularly limited in its configuration, standard, or combination as long as it can send and receive data.

The client terminal 102 is a terminal used by the user, and may be, for example, an information processing terminal such as a PC (Personal Computer) or a mobile terminal such as a smartphone or tablet terminal.

The food management server 103, the data collection server 104, and the estimation server 105 are each running on the server computer. The food management server 103 is a server that holds food data of agricultural products and customer data of food purchasers shown in Table 1 described later, and provides food data and customer data to the data collection server 104. The data collection server 104 is a server that receives data provided by the food purchaser from the client terminal 102, weights the data, and provides the data to the estimation server 105. The estimation server 105 generates training data using the data passed from the data collection server 104, performs training using the training data, and creates a trained model. In this specification, the model at the time of learning is described as a learning model, and the model obtained as a result of learning is described as a trained model. The estimation process performed by the estimation server 105 is performed using the trained model. When the estimation server 105 receives the deliciousness estimation request from the client terminal 102, the estimation server 105 inputs the input data generated from the image data indicated by the estimation request into the trained model and estimates the deliciousness. Then, the estimation server 105 returns the evaluation value obtained as the estimation result to the client terminal 102 as a response to the estimation request.

Hereinafter, the server functions described in the present specification may be realized by a single server or a single virtual server, or may be realized by a plurality of servers or a plurality of virtual servers. Alternatively, multiple virtual servers may be running on a single server. Further, although each device is shown one by one in FIG. 1, the number is not limited to this, and the number may vary.

[Hardware configuration]
FIG. 2 is a diagram showing an example of a hardware configuration of an information processing device 200 applicable as a client terminal 102, a food management server 103, a data collection server 104, and an estimation server 105 according to the present embodiment. Here, each device is described as having the same configuration, but each device may have a different configuration. For example, in a device that does not perform learning processing, GPU 209, which will be described later, may not be provided.

The CPU (Central Processing Unit) 202 controls the entire device. The CPU 202 executes an application program, an OS (Operating System), etc. stored in the HDD (Hard Disk Drive) 205, and temporarily stores information, files, etc. necessary for executing the program in the RAM (Random Access Memory) 204. Control to do. The GPU (Graphics Processing Unit) 209 performs processing when learning is performed a plurality of times by a machine learning method such as deep learning. By using GPU209, more data can be processed in parallel, and efficient calculation can be performed. The ROM (Read Only Memory) 204 is a storage means, and internally stores various data such as a basic I / O program. The RAM 204 is a temporary storage means, and functions as a main memory, a work area, and the like of the CPU 202 and the GPU 209. The HDD 205 is one of the external storage means, functions as a large-capacity memory, and stores an application program such as a Web browser, a service server group program, an OS, and related programs.

The display unit 208 is a display means such as a display, and displays a command input from an input unit 207 composed of a keyboard, a mouse, or the like, various UI screens, and the like. The system bus 201 controls the flow of data in the device. Each part of the information processing apparatus 200 is communicably connected via the system bus 201. The NIC (Network Interface Card) 206 exchanges data with an external device via various networks. The configuration of the computer is an example thereof, and is not limited to the configuration example of FIG. For example, the storage destination of data and programs can be changed in ROM 203, RAM 204, HDD 205, etc. according to the characteristics. In addition, when the CPU 202 or GPU 209 executes the process based on the program stored in the HDD 205, the process in the software configuration as shown in FIG. 3 is realized.

[Software configuration]
FIG. 3 is a diagram showing an example of the software configuration of the deliciousness estimation system according to the present embodiment. The software configuration shown below is an example, and parts that are not directly related to this embodiment are omitted. In the present embodiment, as an example of food, agricultural products such as apples will be described as an example, but the present invention is not limited to this. For example, it may be seafood such as fish or other fresh food.

The client terminal 102 includes a browser 301 that can access an external server. The type of the browser 301 here is not particularly limited as long as it has a function as a web browser.

The food management server 103 includes a data storage unit 311, a food management unit 312, and a customer management unit 313. The data storage unit 311 records food data (Table 1) of agricultural products managed by the food management unit 312 and customer data (Table 2) managed by the customer management unit 313. Table 1 shows an example of food data of agricultural products managed by the food management department 312.

The food ID column is a management ID (identification information) that uniquely indicates an agricultural product. The variety line is the variety of crops. In the present embodiment, the trained model created by the estimation server 105 is created for each of these varieties. The production area column indicates the production area where the crop was harvested. The shipping date column is the date when the crop was shipped. The best time to eat column is the best time to eat for each variety for which the shipping date has been calculated. The deadline for eating will be described later. When the food management unit 312 receives the food ID from the data collection / provision unit 321 of the data collection server 104, the food management unit 312 refers to the food data as shown in Table 1 and returns the food data corresponding to the food ID.

Table 2 shows an example of customer data which is user information managed by the customer management unit 313.

The customer ID column is a management ID (identification information) that uniquely indicates a customer. The management ID is, for example, a value obtained from an authorization token (not shown) associated with a request from the browser 301 of the client terminal 102. The point column is point return data for each food purchaser that serves as a reward for providing the data. In the present embodiment, points are redeemed in order to promote data provision from the user, but this configuration may be omitted. Alternatively, the configuration may be such that the data provision from the user is promoted by another method. The separation rate column is a value that indicates the ratio of the number of past times that there was a gap between the evaluation value of the data provided by the food purchaser and the evaluation value that estimated the deliciousness by inputting the image data into the trained model. is there. Here, the denominator of the separation rate indicates the number of times the data was provided, and the numerator of the separation rate indicates the number of times the separation occurred. It should be noted that the number of times the gap may occur except when the evaluation value indicated by the data provided by the food purchaser and the evaluation value of the estimation result by the trained model completely match, or a difference of a certain value or more occurs. In this case, the number of times the gap occurs may be used. The separation rate is updated as historical information each time data is provided.

The data collection server 104 includes a data collection / provision unit 321 and a weight determination unit 322. The data collection / provision unit 321 collects various data from the client terminal 102. In addition, the data collection / provision unit 321 provides the collected data to the estimation server 105. The weight determination unit 322 determines the weight value of each data based on the ability to determine the deliciousness of the evaluator. The details of the weight value will be described later.

The estimation server 105 includes a data storage unit 331, a learning data generation unit 332, a learning unit 333, and an estimation unit 334. The data storage unit 331 stores various data provided by the data collection server 104. The learning data generation unit 332 generates learning data used for learning processing by the learning unit 333 from various data stored in the data storage unit 331. The learning unit 333 performs learning processing of the learning model using the learning data generated by the learning data generation unit 332. Further, the learning unit 333 provides the estimation unit 334 with a learned model obtained as a result of the learning process. The trained model used by the estimation unit 334 will be updated as the learning progresses. The estimation unit 334 performs the estimation process using the learned model obtained by the learning process of the learning unit 333. In this embodiment, the estimation process is executed in response to a request from the client terminal 102.

[Learning model]
FIG. 8 is a diagram for explaining the concept of input / output of the learning model according to the present embodiment. A learned model is generated by repeatedly learning the learning model 803 by the learning unit 333. As the trained model, a part of the training model 803 may be used, or the whole may be used. Specific learning algorithms include the nearest neighbor method, the naive Bayesian method, a decision tree, and a support vector machine. As appropriate, any of the above algorithms that can be used can be applied to this embodiment.

The input data 801 for the learning model 803 is food image data which is data acquired by the learning data generation unit 332 from the data collection server 104. The output data 802 is an evaluation value of the deliciousness estimated by the estimation unit 334 by the learning model 803 based on the input data 801. The evaluation value is a value representing the deliciousness of the food indicated by the image data of the input data 801 and is represented here in five stages of "1" to "5". Table 3 shows an example of the definition of the evaluation value. This defined value is managed by the learning data generation unit 332 of the estimation server 105 and recorded in the data storage unit 331. The higher the number, the better the taste. The evaluation information indicating the deliciousness is not limited to this, and the number may be increased or decreased, or the evaluation information may be based on another evaluation method.

For example, according to "Evaluation of the appearance of apples" (published journal IEICE Technical Report: IEICE Technical Report 106 (410) 2006.12.7.8 p.57-61), apples look like apples. When evaluating the deliciousness, research results that are emphasized in the order of surface condition, color, and shape are introduced. In the deliciousness estimation system according to the present embodiment, the image data of the food whose surface condition, color, and shape characteristics are known is used as the input data of the learning model 803, and the evaluation value of the deliciousness of the food is output to estimate the deliciousness. do.

[Outline of operation]
FIG. 4 is a diagram showing an outline of the operation of the deliciousness estimation system according to the present embodiment. In the deliciousness estimation system according to the present embodiment, the receipt provided to the food purchaser at the purchasing store or the like includes the food purchase information, the access information to the data collection server 104, and the food ID indicating the food data as a two-dimensional code. Describe. The method of displaying the above information may be provided by another method. (1) to (11) in FIG. 4 correspond to the learning phase, and (12) to (14) in FIG. 4 correspond to the estimation phase. Each phase may be performed independently.

The food purchaser who provides the data used for learning to the deliciousness estimation system according to the present embodiment reads the two-dimensional code of the receipt with the browser 301 of the client terminal 102 and sends it to the data collection server 104 via the browser 301. to access. In the browser 301 of the client terminal 102, UI screens 901 to 903, which will be described later, are displayed with reference to FIG. 9 in order to input the deliciousness evaluation value of the food provided by the data collection server 104 and the image data of the food.

In FIG. 4 (1), the data collection server 104 receives various data from the browser 301. The data received here includes food image data, food purchaser's evaluation value, tongue image data, occupational information, food ID, and the like. In FIG. 4 (2), the data collection server 104 acquires food data and customer data from the food management server 103 using the food ID and the customer ID as keys.

FIG. 4 (3), the data collection / provision unit 321 of the data collection server 104 analyzes the image of the data acquired from the client terminal 102 and determines the type of food. If the varieties in the food data (Table 1) do not match the determined varieties, this processing sequence is terminated. The operation of FIG. 4 (3) corresponds to the process of S503 of FIG. 5 described later.

In FIG. 4 (4), the data collection / provision unit 321 determines whether the date of receiving the data from the client terminal 102 has passed the deadline for eating the food data (Table 1). If the deadline for eating has passed, this data will not be used as learning data. The deadline for eating may be set by the food producer or calculated from the shipping date. In addition, the expiration date for eating may be obtained from the expiration date or the expiration date. The operation of FIG. 4 (4) corresponds to the process of S504 of FIG. 5 described later. In addition, the data collection / provision unit 321 passes the customer ID to the food management unit 312 and acquires customer data (Table 2).

In FIG. 4 (5), the weight determination unit 322 of the data collection server 104 determines the weight value of the corresponding data for the ability to determine the deliciousness of the evaluator. The ability to judge the taste of the evaluator is a value indicating the ability of the food purchaser who evaluated the taste to evaluate the taste. In this embodiment, as the evaluator ability, as shown in the reference value of the evaluator ability (Table 4), three types of occupation, taster value, and separation rate are used. The total value of the weight for the occupation, the weight of the taster value, and the weight of the separation rate, which are the weight values of each of the three types, becomes the weight value of the data.

The weight value is determined from the occupation information received from the browser 301 of the client terminal 102 together with various data, the image data of the tongue, or the separation rate detected from the data provided in the past. The weight determination unit 322 holds a reference value (Table 4) of the evaluator ability as a reference when determining the weight. The standard values for evaluator abilities are shown below.

The method of determining the weight for each type column will be described. Regarding the weight for occupation, the weight is defined as a reference value for each occupation value in the customer data. The taster value is determined by the weight determination unit 322 from the image data of the tongue. As a mechanism of taste in molecular biology, five basic taste signals of sweetness, saltiness, bitterness, acidity, and umami contained in chemical substances received by taste receptors at the tip of taste buds in the papillary of the tongue are central nerves. It is said that a person feels the taste and judges the taste by being perceived by the brain through the passage. The taster value according to the present embodiment is a value indicating the density of taste buds in the papilla of the tongue in a certain area, and is a value obtained from the image data of the tongue provided by the food purchaser together with the learning data. That is, the taster value is used as information on the sensitivity to taste. The weight determination unit 322 determines the number of taste buds in a certain area from the image data of the tongue, and defines the taster value as a value indicating the density. The determination method here is not particularly limited, and may be performed based on a known image analysis method. As for the separation rate, the weight determination unit 322 calculates the weight of the food purchaser with respect to the separation rate of the customer data (Table 2) acquired through the data collection / provision unit 321.

The types used for the evaluator ability are not limited to the above three types, and other information may be used. For example, the evaluator's vital information (blood pressure, pulse, respiratory rate, body temperature, heart rate, etc.) may be further included. Alternatively, information such as the food management status and the climate at the time of evaluation may also be used. Further, other values may be used as the particle size and the threshold value for each type.

When the weight determination unit 322 determines the weight of the learning data, the data collection / provision unit 321 transmits various data and weight information to the learning data generation unit 332 of the estimation server 105 in FIG. 4 (6). When the learning data generation unit 332 receives various data, the learning data generation unit 332 records the data in the data storage unit 331.

In FIG. 4 (7), the learning data generation unit 332 generates image data for weight information from the image data received from the data collection / provision unit 321. Create different image data with the same features by rotating or scaling the image of the image data. The method for creating image data having a plurality of patterns here is not limited to rotation and scaling, and other methods may be used. This corresponds to the process of S603 of FIG. 6 described later. Then, the learning data generation unit 332 transmits the learning data in which the evaluation values of the respective data are set for the image data to the learning unit 333.

In FIG. 4 (8), the learning unit 333 performs learning by applying the image data and the evaluation value to the learning model 803 for each variety. As a result of the learning process, a trained model is created. This corresponds to the process of S605 of FIG. 6 described later. By repeating this for the weight information, more learning data provided by the evaluator with high ability is input to the learning model 803, and a highly accurate trained model is created.

In FIG. 4 (9), the estimation unit 334 performs the estimation process using the trained model generated in FIG. 4 (8). The input data here may be the original image of the image data used for learning.

In FIG. 4 (10), the estimation unit 334 notifies the food management server 103 of the estimation result of FIG. 4 (9). The customer management unit 313 of the food management server 103 updates the separation rate of the customer information (Table 2) based on the estimation result received from the estimation server 105.

In FIG. 4 (11), the food management server 103 adds a predetermined point to the point sequence of the customer information (Table 2) in response to the acquisition of the estimation result from the estimation server 105. The content of the points to be added here is not particularly limited.

In FIG. 4 (12), the estimation unit 334 of the estimation server 105 receives a deliciousness estimation request from the browser 301 of the client terminal 102. In the browser 301 of the client terminal 102, screens 911 to 913, which will be described later, are displayed with reference to FIG. 9 in order to make an estimation request.

In FIG. 4 (13), the estimation unit 334 inputs the image data indicated by the estimation request into the trained model and estimates the deliciousness.

In FIG. 4 (14), the estimation unit 334 returns the estimated deliciousness evaluation value to the browser 301 of the client terminal 102.

[Processing flow]
(Processing in the learning phase)
The processing flow of the learning phase of the deliciousness estimation system according to the present embodiment will be described with reference to FIGS. 5 and 6.

FIG. 5 is a processing flow in the learning phase of the data collection server 104. This processing flow is realized by the CPU 202 of the information processing device 200 operating as the data collection server 104 reading and executing the program stored in the HDD 205 or the like.

In S501, the data collection server 104 determines whether or not a registration request including various data has been received from the browser 301 of the client terminal 102. The registration request here includes food image data, food purchaser's evaluation value, tongue image data, occupational information, food ID, and the like. The information included in the registration request here is input on the UI screens 901 to 903 of FIG. 9, which will be described later. When the registration request is received (YES in S501), the process proceeds to S502, and when the registration request is not received (NO in S501), the process waits until it is received.

In S502, the data collection server 104 obtains the customer ID from the authorization token (not shown) indicated in the registration request. Further, the data collection server 104 passes the food ID and the customer ID indicated in the registration request to the food management server 103, and acquires the food data (Table 1) and the customer data (Table 2). Further, the data collection server 104 analyzes the image data of the food indicated in the registration request and identifies the food variety.

In S503, the data collection server 104 determines whether the variety of the food data (Table 1) corresponding to the designated food ID matches the variety specified from the image data in S502. If they match, the process proceeds to S504 (YES in S503), and if they do not match (NO in S503), the present processing flow ends.

In S504, the data collection server 104 refers to the food data (Table 1) and confirms whether or not the eating period of the target food has been exceeded. If it does not exceed (YES in S504), the process proceeds to S505, and if it exceeds (NO in S504), the present processing flow ends.

In S505, the data collection server 104 obtains the taster value from the tongue image data acquired from the browser 301 as the evaluator ability. Further, the data collection server 104 determines the weight based on the reference value (Table 4) of the evaluator ability from the occupation information acquired from the browser 301 as the evaluator ability, the taster value, and the separation rate acquired from the customer data. ..

In S506, the data collection server 104 transmits various data and weight information to the estimation server 105. Then, this processing flow is terminated.

FIG. 6 is a processing flow in the learning phase of the estimation server 105. This processing flow is realized by the CPU 202 and GPU 209 of the information processing device 200 operating as the estimation server 105 reading and executing the program stored in the HDD 205 and the like.

In S601, the estimation server 105 receives various data and weight information from the data collection server 104. This information is the information transmitted from the data collection server 104 in S506 of FIG.

In S602, the estimation server 105 determines whether the value of the variable N is smaller than the value of the weight information received in S601. The initial value of the variable N is set to 0, and is incremented by 1 each time the determination is made. If N <weight, the process proceeds to S603 (YES in S602), and if N ≥ weight (NO in S602), the process proceeds to S607.

In S603, the estimation server 105 rotates or scales the image data received in S601 to create image data for learning.

In S604, the estimation server 105 inputs the image data and the evaluation value created in S603 into the learning model 803 as learning data. The evaluation value here is used as teacher data for the evaluation value output from the learning model 803.

In S605, the estimation server 105 performs learning using the learning data input in S604. For example, the estimation server 105 sets the parameters of the learning model 803 so that the difference between the evaluation value, which is the output data output to the image data input to the learning model 803, and the evaluation value, which is the teacher data, becomes small. Learning is performed by controlling.

In S606, the estimation server 105 adapts the trained model obtained as a result of the learning process in S605 to the estimation unit 334. In the adaptation here, for example, for each food, a part or all of the learning model 803 is reflected in the estimation unit 334 as a learned model. After that, the process returns to S602, the variable N is updated, and the determination is performed again. That is, the processes of S603 to S606 are repeated by the amount of the weight information received in S601.

In S607, the estimation server 105 inputs the image data acquired from the data collection server 104 in S601 into the trained model, and executes the estimation process.

In S608, the estimation server 105 notifies the food management server 103 of the estimation result of the estimation process in S607. Then, this processing flow is terminated.

Although S602 in the present embodiment shows an example in which the number of learnings and the number of weights are set to the same value for repeated learning, the values of the number of learnings and the number of weights do not necessarily have to be the same. That is, even if the values have different learning counts and weight counts, they may have a positive correlation. Specifically, if the weight value is 1 to 10, the number of learnings is one, if the weight value is 11 to 20, the number of learnings is three, and if the weight value is 21 to 30, learning is performed. The number of times may be such that each increases, such as 5 times.

(Processing in the estimation phase)
The processing flow of the estimation phase of the deliciousness estimation system according to the present embodiment will be described with reference to FIG. 7. The processing flow of FIG. 7 is realized by the CPU 202 and GPU 209 of the information processing device 200 operating as the estimation server 105 reading and executing the program stored in the HDD 205 and the like.

In S701, the estimation server 105 determines whether or not an estimation request including food image data has been received from the browser 301 of the client terminal 102. The estimation request here is transmitted from the client terminal 102 based on the information input using the UI screens 911 to 913 of FIG. 9 described later. When the image data is received (YES in S701), the process proceeds to S702, and when the image data is not received (NO in S701), the process waits until the image data is received.

In S702, the estimation unit 334 of the estimation server 105 inputs the image data indicated by the estimation request received in S701 into the trained model.

In S703, the estimation unit 334 acquires a value (evaluation value) indicating the deliciousness output from the trained model as an estimation result for the image data input in S702.

In S704, the estimation server 105 transmits the evaluation value acquired in S703 to the browser 301. Then, this processing flow is terminated.

[UI screen]
FIG. 9 shows a configuration example of the UI screen of the deliciousness estimation system according to the present embodiment. Each UI screen is displayed on the browser 301 of the client terminal 102. Here, the client terminal 102 will be described as a mobile terminal.

The UI screens 901 to 903 show an example of a UI used by a food purchaser and an evaluator who evaluates the taste of food to provide data for estimating the taste. The UI screens 901 to 903 are displayed by the response that the browser 301 of the client terminal 102 accesses the data collection server 104, and are used before the data transmission of FIG. 4 (1). The UI screen 901 is an example of a UI screen for selecting the target food of the data to be provided. In the area 904, the mark is displayed by picking up the image portion of the food that can be the target food from the image data acquired by the camera function (not shown) of the client terminal 102. If the OK button 905 is pressed after the mark is displayed, the screen transitions to the next screen. For example, when there is only one target food, the UI screen 902 is displayed. When there are a plurality of target foods, the transition may be made to a UI screen (not shown) for allowing the operator to specify the target product.

The UI screen 902 is an example of a UI screen for recording the evaluation value, occupation information, and tongue image data of the evaluator in Table 3 of the evaluator. The area 906 displays an image of the food to be evaluated selected in the area 904 of the UI screen 901. The area 907 is an area for inputting an evaluation value and occupation information. When the tongue image capturing button 908 is pressed, the UI screen 903 is displayed. When the transmission button 909 is pressed, the image data including the target food selected by the evaluator, the evaluation value, the occupation information, and the tongue image data are transmitted from the client terminal 102 to the data collection server 104.

The UI screen 903 is a UI screen for capturing the tongue image data of the user. After taking a tongue image using the camera function (not shown) provided in the client terminal 102, the tongue image data to be transmitted can be selected by pressing the enter button 910.

The UI screens 911 to 913 show an example of a UI used when the person who is planning to purchase food and estimates the taste of food. The UI screens 911 to 913 are displayed by the response that the browser 301 of the client terminal 102 accesses the estimation server 105, and are used before making the estimation request of FIG. 4 (12). The UI screen 911 is an example of a UI screen for selecting a food to be estimated for deliciousness. In the area 914, the mark is displayed by picking up the image portion of the food that can be the target food from the image data acquired by the camera function (not shown) of the client terminal 102. When the send button 915 is pressed, the estimation request is transmitted to the estimation server 105. If the OK button 915 is pressed after the mark is displayed, the next UI screen is displayed. For example, when there is only one food for which the taste is estimated, the UI screen 912 is displayed. When there are a plurality of target foods, the transition may be made to a UI screen (not shown) for allowing the operator to specify the target product.

The UI screen 912 is a UI screen for sending image data of food to be estimated for deliciousness to the estimation server 105. When the send button 916 is pressed, the estimation request is transmitted to the estimation server 105. The UI screen 913 is an example of a UI screen for displaying the estimated food image data and the estimated deliciousness evaluation value as a result of the estimation process by the estimation server 105. The evaluation value as the estimation result is displayed in the area 917.

As described above, in the present embodiment, the evaluator ability of deliciousness is weighted, each data provided by the evaluator with high ability is given a high weight, and the number of times data corresponding to the weight is trained, and the trained model is trained. To generate. Therefore, it is possible to generate a trained model in which more training is performed using data having a high weight and a highly accurate estimation result can be obtained. Therefore, it is possible to provide a highly accurate taste estimation system.

<Other Embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. But it is feasible. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

102 ... client terminal, 103 ... food management server, 104 ... data collection server, 105 ... estimation server

Claims (10)

It is a deliciousness estimation system that estimates the deliciousness of food.
Acquisition means for acquiring data including image data of food, evaluation information of deliciousness of the food, and user information of the user who evaluated the food,
A determination means for determining the weight for the data based on the user information contained in the data, and
A generation means for generating one or more image data for learning from the image data included in the data according to the weight determined by the determination means, and a generation means.
Learning to estimate the deliciousness of food by performing learning processing using one or more image data for learning generated by the generation means as input data and evaluation information included in the data as teacher data. A deliciousness estimation system characterized by having a learning means for generating a completed model. A receiving means for receiving an estimation request including image data of food, and
The deliciousness estimation according to claim 1, further comprising an estimation means for estimating the deliciousness of the food by inputting the image data of the food included in the estimation request into the trained model. system. The deliciousness estimation system according to claim 1 or 2, wherein the generation means generates one or a plurality of image data for learning by rotating or scaling the image data included in the data. The deliciousness according to any one of claims 1 to 3, wherein the generation means generates more image data for learning from the image data included in the data as the value of the weight increases. Data estimation system. The deliciousness estimation system according to any one of claims 1 to 4, wherein the user information includes information on the occupation of the user and image data of the tongue of the user. The determination means further includes a specific means for specifying the taste sensitivity of the user from the image data of the tongue of the user.
The deliciousness estimation system according to claim 5, wherein the determination means determines the weight based on the sensitivity of the taste specified by the specific means. The determination means includes the evaluation information of the deliciousness of the food evaluated by the user and the estimation result obtained by inputting the image data of the food into the trained model generated by the learning means as input data. The deliciousness estimation system according to any one of claims 1 to 6, wherein the weight is determined based on the history of the distance. It further has a management means for managing the deadline for food consumption and a determination means for determining whether or not the food indicated by the data acquired by the acquisition means has exceeded the deadline.
The deliciousness according to any one of claims 1 to 7, wherein the learning means does not use the data determined by the determination means to exceed the deadline as learning data. Estimate system. It is a control method of the deliciousness estimation system that estimates the deliciousness of food.
An acquisition process for acquiring data including image data of food, evaluation information of deliciousness of the food, and user information of the user who evaluated the food, and
A determination process for determining weights for the data based on the user information contained in the data, and
A generation step of generating one or more image data for learning from the image data included in the data according to the weight determined in the determination step, and a generation step.
Learning to estimate the deliciousness of food by performing learning processing using one or more image data for learning generated in the generation step as input data and evaluation information included in the data as teacher data. A control method characterized by having a learning process for generating a completed model. Computer,
Acquisition means for acquiring data including image data of food, evaluation information of deliciousness of the food, and user information of the user who evaluated the food,
A determinant that determines the weight for the data based on the user information contained in the data.
A generation means for generating one or more image data for learning from the image data included in the data according to the weight determined by the determination means.
Learning to estimate the deliciousness of food by performing learning processing using one or more image data for learning generated by the generation means as input data and evaluation information included in the data as teacher data. A program to function as a learning means to generate an input model.

Images