JP2022111052A - Vegetable and fruit quality estimation system - Google Patents

Abstract

To highly accurately and automatically estimate qualities of vegetables and fruits without depending on more manpower.SOLUTION: A vegetable and fruit quality estimation system previously acquires three or more levels of association degrees between reference appearance information and qualities of vegetables and fruits for individual portions of the vegetables and fruits and estimates the qualities of the vegetables and the fruits by reading out the association degrees of the portions of the vegetables and fruits in accordance with a first imaging target portion information which has been acquired, when: imaging appearance information related to appearances of the vegetables and fruits and acquiring the first imaging target portion information related to the imaged portions of the vegetable and fruits; and determining the three or more levels of association degrees between the reference appearance information obtained by imaging of the appearance of the vegetables and fruits in the past and the qualities of the vegetables and fruits, using a learned model in which the reference appearance information is input and the qualities of the vegetables and fruits are output, prioritizing a higher association degree on the basis of the reference appearance information identical with or similar to the appearance information acquired by information acquisition means, and estimating the qualities of the vegetables and fruits.SELECTED DRAWING: Figure 19

Description

The present invention relates to a fruit and vegetable quality estimation system suitable for estimating the quality of fruit and vegetables with high accuracy.

The quality of fruits and vegetables is evaluated from the viewpoint of taste such as texture and taste (sweetness, sourness, aroma), in addition to appearance information such as the size, shape, color and degree of damage of fruits and vegetables. be. The quality of such fruits and vegetables has been evaluated through human visual inspection in terms of appearance, and has been evaluated through tasting in terms of texture and taste.

However, if the evaluation of the quality of these fruits and vegetables is all done manually, the burden of labor will increase, and considerable skill and experience will be required for discrimination accuracy. When a veteran quality evaluator retires, it is necessary to find a successor who can maintain the accuracy of the evaluation.

For this reason, there has been a demand for a system that can evaluate the quality of fruits and vegetables with high accuracy without relying on human labor.

Therefore, the present invention has been devised in view of the above-mentioned problems, and the object thereof is to provide a method for estimating the quality of fruits and vegetables with high accuracy and automatically without relying on human labor. It is to provide a quality estimation system.

A fruit and vegetable quality estimation system according to the present invention is a fruit and vegetable quality estimation system for estimating the quality of fruits and vegetables. Information acquisition means for acquiring first imaging target part information related to the imaged parts of the fruits and vegetables, reference appearance information obtained by imaging the appearance of the fruits and vegetables in the past, and quality of the fruits and vegetables. Based on the same or similar reference appearance information as the appearance information acquired by the information acquisition means, using a trained model in which the degree of association is specified, the input is the reference appearance information, and the output is the quality of the fruits and vegetables estimating means for estimating the quality of the fruits and vegetables by prioritizing those having a higher degree of association; and second imaging target part information relating to parts of the fruits and vegetables contained in the reference appearance information is linked to the reference appearance information. The user terminal stores the first imaging target region information acquired by the information acquisition means and the reference appearance information identical or similar to the appearance information acquired by the information acquisition means. and the second imaging target region information recorded in the recording means in association with the user, and notifying the user of the suggestion of the imaging method.

It is possible to automatically estimate the quality of fruits and vegetables with high accuracy without relying on human labor.

1 is a block diagram showing the overall configuration of a system to which the present invention is applied; FIG. It is a figure which shows the specific structural example of a search apparatus. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention. It is a figure for demonstrating the operation|movement of this invention.

First Embodiment Hereinafter, examples of a fruit and vegetable quality estimation model construction system and a fruit and vegetable quality estimation system to which the present invention is applied will be described in detail with reference to the drawings.

Fruits and vegetables in the following examples include all fruits and vegetables. The fruits mentioned below include all fruits on the earth, but among them, in particular, stone fruits (apricots, plums, cherries, plums, nectarines, prunes, peaches, etc.), kernel fruits (pears, pears, loquats, apples), tropical fruits (avocado, kiwifruit, durian, papaya, mango, etc.), fruity vegetables (strawberries, watermelons, pineapples, bananas, melons), grapes (grapes, giant grapes, muscat grapes, blueberries), persimmons, chestnuts The present invention is suitable for estimating the quality of In the following description, when referring to fruit, a case will be described in which any kind and variety of all fruits on the earth are targeted. The types of fruits referred to here are (means types such as melons, mangoes, pears, cherries, etc.). , ``Chojuro'', ``Kikusui'', and ``Nijisseiki''.

Vegetables include peppers, asparagus, eggplant, green beans, cowpea, okra, corn, long onions, onions, Japanese mustard spinach, shallots, garlic, bamboo shoots, shiitake mushrooms, matsutake mushrooms, mushrooms, turnips, pumpkins, edible chrysanthemums, cabbage, cucumbers, Kyoto greens ( mizuna), green peas, garland chrysanthemum, spinach, kuwai, burdock root, tomato, perilla (perilla), juniper, ginger, melon, cucumber, zucchini, carrot, lettuce, sweet potato, potato, cauliflower, broccoli, celery, radish, mustard mustard, sugarcane, etc. is mentioned as a typical one, but it is not limited to this, and any other vegetables are included.

In the following description, as a fruit and vegetable quality estimation program to which the present invention is applied, an apple quality estimation program focusing on apples as fruits and vegetables will be described as an example, but other fruits and vegetables other than apples also use the same method. Of course, quality estimation can be realized by doing so.

FIG. 1 is a block diagram showing the overall configuration of an apple quality estimation system 1 in which an apple quality estimation program to which the present invention is applied is implemented. The apple quality estimation system 1 includes an information acquisition section 9 , an estimation device 2 connected to the information acquisition section 9 , and a database 3 connected to the estimation device 2 .

The information acquisition unit 9 is a device for a person using this system to input various commands and information. The information acquisition unit 9 is not limited to a device for inputting text information, and may be configured by a device such as a microphone that can detect voice and convert it into text information. Further, the information acquisition unit 9 may be configured as an imaging device capable of capturing an image, such as a camera. The information acquisition unit 9 may be configured by a scanner having a function of recognizing a character string from a paper document. Further, the information acquisition unit 9 may be integrated with the estimation device 2 to be described later. The information acquisition unit 9 outputs the detected information to the estimation device 2 . Further, the information acquisition unit 9 may be configured by means for specifying position information by scanning map information. Further, the information acquisition unit 9 may be configured with an illuminance sensor for measuring a temperature sensor, a humidity sensor, and a wind direction sensor. The information acquisition unit 9 may also be configured with a communication interface that acquires weather data from the Meteorological Agency or a private weather forecast company. The information acquisition unit 9 may be composed of a body sensor worn on the body to detect body data, and the body sensor detects, for example, body temperature, heart rate, blood pressure, number of steps, walking speed, and acceleration. It may be configured with a sensor for The information acquisition unit 9 may be configured as a device that acquires information such as drawings by scanning or reading from a database. The information acquisition unit 9 may be configured by an odor sensor that detects odors and scents in addition to these.

The database 3 accumulates various information and learning data necessary for apple quality estimation. Information necessary for estimating apple quality includes reference appearance information on the appearance of apples acquired in the past, reference variety information on apple varieties when acquiring reference appearance information, and acquisition of reference appearance information. Spectral information for reference obtained from apples when obtaining reference scent information for reference about the scent of apples when obtaining appearance information for reference Reference weight information about the weight of apples when obtaining appearance information for reference for reference A data set of reference cultivation history information relating to the cultivation history of apples at the time of acquiring the appearance information and the quality of apples for which judgments have actually been made is stored.

In other words, in addition to such learning data and reference appearance information, the database 3 stores at least one of reference variety information, reference spectrum information, reference fragrance information, reference weight information, and reference cultivation history information. , and the quality of apples are linked to each other and stored. Note that instead of storing such learning data in the database 3, the learning data may be stored in the estimation device 2 or the information acquisition unit 9 side. That is, the estimating device 2 and the information acquiring unit 9 may be so-called edge devices, and the edge devices may be equipped with artificial intelligence to accumulate learning data and perform inference.

The estimating device 2 is composed of an electronic device such as a personal computer (PC), for example, but in addition to the PC, it is embodied in any other electronic device such as a mobile phone, a smartphone, a tablet terminal, a wearable terminal, etc. It may be converted. A user can obtain a search solution by this estimation device 2 .

FIG. 2 shows a specific configuration example of the estimation device 2 . The estimating device 2 performs wired or wireless communication with a control unit 24 for controlling the entire estimating device 2 and an operating unit 25 for inputting various control commands via operation buttons, a keyboard, or the like. A communication unit 26 for searching, an estimating unit 27 for making various judgments, and a storage unit 28, represented by a hard disk, for storing a program for performing a search to be executed are connected to the internal bus 21, respectively. . Further, the internal bus 21 is connected with a display unit 23 as a monitor for actually displaying information.

The control unit 24 is a so-called central control unit for controlling each component implemented in the estimating device 2 by transmitting control signals via the internal bus 21 . In addition, the control unit 24 transmits commands for various controls through the internal bus 21 according to operations through the operation unit 25 .

The operation unit 25 is embodied by a keyboard or a touch panel, and a user inputs an execution command for executing a program. The operation unit 25 notifies the control unit 24 when the execution command is input by the user. Upon receiving this notification, the control unit 24 cooperates with each component including the estimation unit 27 to execute desired processing operations. The operation unit 25 may be embodied as the information acquisition unit 9 described above.

The estimation unit 27 estimates a search solution. The estimation unit 27 reads various information stored in the storage unit 28 and various information stored in the database 3 as necessary information in executing the estimation operation. This estimation unit 27 may be controlled by artificial intelligence. This artificial intelligence may be based on any known artificial intelligence technology.

The display unit 23 is composed of a graphic controller that creates a display image based on control by the control unit 24 . The display unit 23 is implemented by, for example, a liquid crystal display (LCD).

When the storage unit 28 is configured with a hard disk, predetermined information is written to each address based on the control by the control unit 24, and this information is read as necessary. The storage unit 28 also stores a program for executing the present invention. This program is read and executed by the control unit 24 .

The operation of the apple quality estimation system 1 configured as described above will be described.

In the apple quality estimation system 1, it is assumed that, for example, as shown in FIG. 3, three or more degrees of association between the reference appearance information and the apple quality are set in advance. The appearance information for reference is information about the appearance of an apple, which is obtained from image information obtained by imaging the apple, and can be obtained by analyzing the image information. This image may be a moving image as well as a still image. This reference appearance information may specify the appearance of an apple based on any of the size, shape, color, and degree of damage of the apple by analyzing an image of the apple. The appearance of these apples may be determined based on previously learned feature values. At this time, artificial intelligence is used to learn the image data such as the size, shape, color, and degree of damage of the apple, and the quality of the apple. The quality of the apple may be determined by comparing with the learned image data. Further, the appearance information for reference is not limited to apples that have actually been picked and harvested after thinning, and image information of apples that are still on branches before thinning may be obtained.

In such a case, using a predictive model machine-learned using image information and the appearance of any of the size, shape, color, and degree of scratches of the apple as teacher data, the input is the image information, and the output is the above Appearance may be specified based on the result of appearance determination.

The quality of the apples may be judged based on previous experience by the evaluator, or may be judged by actually tasting the apples. In such a case, multiple testers who taste the apples evaluate each item such as sweetness, sourness, softness, hardness, aroma, texture, bitterness, etc. on multiple stages, and the results are statistically evaluated. It may be analyzed systematically and used as a quality evaluation value. Also, the quality of apples may be determined through a taste sensor capable of detecting taste. Also, the quality of apples may be composed of information on the presence or absence of disease, or may be composed of information on the type of disease if a disease has occurred. In addition, as a search solution, instead of the quality of apples, diseases occurring in apples may be applied. This makes it possible to estimate diseases occurring in apples by a similar procedure.

In the example of FIG. 3, it is assumed that the input data are, for example, reference appearance information P01 to P03. Such reference appearance information P01 to P03 as input data are linked to the quality of apples as output. In this output, the apple quality is displayed as the output solution.

The reference appearance information is associated with each other through three or more degrees of association with the quality A to D of the apple as the output solution. Appearance information for reference is arranged on the left through this degree of association, and the quality of each apple is arranged on the right through the degree of association. The degree of relevance indicates the degree of relevance to the quality of which apple with respect to the reference appearance information arranged on the left side. In other words, the degree of association is an index indicating which quality of apple each item of reference appearance information is likely to be associated with, and is useful for selecting the most probable apple quality from the reference appearance information. It shows the accuracy in In the example of FIG. 3, w13 to w19 are shown as association degrees. These w13 to w19 are shown in 10 stages as shown in Table 1 below, and the closer to 10 points, the higher the degree of correlation between each combination as an intermediate node and the quality of the apple as an output. Conversely, the closer to one point, the lower the degree of correlation between each combination as an intermediate node and the price as an output.

The estimating device 2 acquires in advance the degrees of association w13 to w19 of three or more stages shown in FIG. In other words, the estimating device 2 accumulates a past data set to determine which of the reference appearance information and the quality of the apple in that case was adopted and evaluated in determining the actual search solution. By analyzing and analysing, the degree of association shown in FIG. 3 is created.

For example, it is assumed that the reference appearance information is α in which the size, shape, color, degree of damage, etc. of the appearance of an apple are evaluated using a plurality of numerical values. It is assumed that the apple quality A is highly evaluated as the apple quality with respect to such reference appearance information. By collecting and analyzing such a data set, the degree of association with the reference appearance information is strengthened.

This analysis may be performed by artificial intelligence. In such a case, for example, in the case of the reference appearance information P01, analysis is performed from various data obtained as a result of evaluating the quality of apples in the past. In the case of this reference appearance information P01, if there are many cases of apple quality A, the degree of association leading to the evaluation of this apple quality is set higher, and if there are many cases of apple quality B , to set a higher degree of association leading to an evaluation of the quality of this apple. For example, in the example of the reference appearance information P01, apple quality A and apple quality C are linked. is set to 2 points.

Further, the degree of association shown in FIG. 3 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence.

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data through a data set of images of the appearance of apples to be evaluated in the past and the quality of apples that are actually estimated and evaluated, the quality of apples is actually newly determined from now on. Above, the quality of the apple will be searched using the learned data described above. In such a case, new appearance information is actually acquired in the area to be determined. Appearance information to be newly acquired is input by the information acquisition unit 9 described above. Appearance information is obtained by taking an image from which the quality of the apple is to be determined. This determination method may be performed by a method similar to that for the reference appearance information described above.

The quality of the apple is determined based on the appearance information newly acquired in this way. In such a case, reference is made to the degree of association shown in FIG. 3 (Table 1), which has been acquired in advance. For example, when the newly acquired appearance information is the same as or similar to P02, the apple quality B is associated with w15 and the apple quality C is associated with the association degree w16. In such a case, the apple quality B with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality C, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

In this way, it is possible to search for the most suitable apple quality from the newly acquired appearance information and display it to the user. By looking at the results of this search, users, namely apple growers, sellers, and distributors, can sort apples based on the quality of the searched apples, and can predict the taste of apples. In addition, you can determine the price of apples. Moreover, since the quality of these products can be predicted through non-destructive inspection without touching the apples, the apples will not be damaged, and the yield can be improved.

Note that when the appearance of the apple before thinning is learned as the reference appearance information, the appearance information may also be acquired by imaging the appearance of the apple to be estimated before the blossoms are picked. This makes it possible to estimate the quality of the apple in the state before fruit thinning on the tree. At this time, the quality of each apple may be associated with harvest time information. Harvest time information here is information indicating when to harvest, and includes, for example, when to harvest, such as whether to harvest now, two days later, four days later, or one week later. Then, after estimating the quality of the apples, the associated harvest time status is displayed. This allows the user to know when to harvest apples before flowering.

In such a case, instead of the harvest time information, the appropriateness of harvest linked to the apple quality may be learned. The suitability for harvesting is information indicating whether harvesting can be done immediately or not.

At this time, when the information acquisition unit 9, or the information acquisition unit 9 and the estimation device are configured by a glasses-type terminal or a head-mounted display (HMD), the apples before thinning which are the trees to be identified through the HMD are included. Appearance information may be obtained by imaging as follows. Then, the quality of the apple as the obtained search solution or the harvest time information linked thereto may be displayed on the display of the HMD. In such a case, so-called AR (Augmented Reality) technology may be used to display information on proposed flowers to be picked on a transparent display. As a result, in the image of the apples before thinning that are visually recognized on the screen, which apples should be harvested immediately, including the suitability of harvesting, can be displayed in a transparent state, and the convenience of the user's work can be displayed. can be improved.

In the example of FIG. 4, it is assumed that a combination of the reference appearance information and the reference product type information is formed. The reference cultivar information is cultivar information related to apple cultivars.

The apple quality depends on the apple variety, in addition to the reference appearance information. Apple varieties include ``Fuji'', ``Jonagold'', and ``Tsugaru'', but the expected taste is different for each variety. In other words, "Fuji" has a crunchy texture expected from "Fuji" and has a balance of sweetness and sourness, all of which are correlated with the appearance image (color, size, etc.). Therefore, by defining the quality of apples in combination with such reference variety information in addition to the reference appearance information, the quality of apples can be evaluated with high accuracy. The reference cultivar information may be obtained by interviewing the producers of the apples for which the reference appearance information is to be obtained, and inputting data on the cultivar names individually, or may be selected and set in advance on the system.

In the example of FIG. 4, it is assumed that the input data are, for example, reference appearance information P01 to P03 and reference product type information P14 to P17. The intermediate node shown in FIG. 4 is a combination of the reference appearance information as input data and the reference product type information. Each intermediate node is also connected to an output. In this output, the apple quality is displayed as the output solution.

Each combination (intermediate node) of the reference appearance information and the reference cultivar information is associated with each other through three or more degrees of association with the apple quality as the output solution. The appearance information for reference and the reference cultivar information are arranged on the left side through the degree of association, and the quality of apples is arranged on the right side through the degree of association. The degree of relevance indicates the degree of high relevance to the quality of apples with respect to the reference appearance information and the reference cultivar information arranged on the left side. In other words, the degree of association is an index indicating to which apple quality each reference appearance information and reference cultivar information is likely to be linked. It indicates accuracy in selecting the most probable apple quality. Therefore, the optimal apple quality is searched for by combining the reference appearance information and the reference variety information.

In the example of FIG. 4, w13 to w22 are shown as association degrees. These w13 to w22 are shown in 10 stages as shown in Table 1. The closer to 10 points, the higher the degree of correlation between each combination as an intermediate node and the output. The closer it is, the lower the degree of correlation between each combination as an intermediate node and the output.

The estimating device 2 acquires in advance the degree of association w13 to w22 of three or more stages shown in FIG. In other words, the estimating device 2 accumulates past data as to which of the reference appearance information, the reference variety information, and the quality of the apple in that case was suitable for determining the actual search solution. , and by analyzing these, the degree of association shown in FIG. 4 is created.

For example, it is assumed that the appearance information for reference in an actual case in the past is the appearance α. It is also assumed that the reference product type information is "Tsugaru". In such a case, the quality of the apple, which indicates the actual quality of the apple, is learned as a data set and defined in the form of the degree of association described above. Such reference appearance information and reference product type information may be extracted from a management database managed by manufacturers, distributors, distributors, and the like.

This analysis may be performed by artificial intelligence. In such a case, for example, when the reference appearance information P01 is the reference variety information P16, the quality of the apple is analyzed from the past data. If there are many cases of apple quality A (sweetness degree, sourness degree, bitterness degree, chewy texture, etc.), the degree of association leading to this quality A is set higher, When there are many cases of quality B and few cases of quality A of apples, the degree of association leading to quality B of apples is set high and the degree of association leading to quality A of apples is set low. For example, in the example of the intermediate node 61a, the outputs of apple quality A and quality B are linked. is set to 2 points.

Further, the degree of association shown in FIG. 4 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence.

In the example of the degree of association shown in FIG. 4, the node 61b is a node of the combination of the reference appearance information P01 and the reference variety information P14. The association degree is w16. A node 61c is a node of a combination of reference position information P15 and P17 with respect to reference appearance information P02, and the degree of relevance of apple quality B is w17, and the degree of relevance of apple quality D is w18. .

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data, the above-described learned data will be used when actually judging the quality of apples. In such a case, appearance information and product type information are actually acquired. For the product type information, the quality of the ring whose quality is to be actually estimated is entered or selected.

Based on the appearance information and variety information newly acquired in this way, the optimum apple quality is searched for. In such a case, reference is made to the degrees of association shown in FIG. 4 (Table 1) that have been acquired in advance. For example, when the newly acquired appearance information is the same as or similar to P02 and the product type information is P17, the node 61d is associated via the degree of association, and this node 61d is associated with apple quality C at w19 and apple quality D at w20. In such a case, the apple quality C with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality D, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

Table 2 below shows examples of degrees of association w1 to w12 extending from the input.

Intermediate nodes 61 may be selected based on degrees of association w1 to w12 extending from this input. In other words, the greater the degree of association w1 to w12, the heavier the weight in selecting the intermediate node 61 may be. However, the degrees of association w1 to w12 may all have the same value, and the weighting in selecting the intermediate nodes 61 may all be the same.

In FIG. 5, in addition to the above-described reference appearance information, three or more levels of association are set between the combination of the reference spectral information instead of the above-described reference cultivar information, and the apple quality for the combination. shows an example.

This reference spectral information added as an explanatory variable instead of the reference positional information is infrared, far-infrared, visible light, near-ultraviolet, and ultraviolet spectral information obtained from apples. Obtaining spectral information from apples is already known to be related to apple quality (taste, gloss). For this reason, the purpose is to improve the accuracy of judging the quality of apples by combining the spectral information for reference to form a degree of association.

In the example of FIG. 5, it is assumed that the input data are, for example, reference appearance information P01 to P03 and reference spectrum information P18 to P21. An intermediate node shown in FIG. 5 is a combination of reference appearance information as input data and reference spectrum information. Each intermediate node is also connected to an output. In this output, the apple quality is displayed as the output solution.

Each combination (intermediate node) of the reference appearance information and the reference spectral information is associated with the apple quality as the output solution through three or more degrees of association. Appearance information for reference and spectral information for reference are arranged on the left side through this degree of association, and apple quality is arranged on the right side through the degree of association. The degree of relevance indicates the degree of high relevance to the quality of the apple with respect to the reference appearance information and the reference spectral information arranged on the left side. In other words, the degree of association is an index indicating the high possibility that each piece of reference appearance information and reference spectrum information is associated with the quality of an apple. It indicates accuracy in selecting the most probable apple quality.

The estimating device 2 acquires in advance the degrees of association w13 to w22 of three or more stages shown in FIG. In other words, the estimating device 2 determines which of the reference appearance information, the reference spectrum information obtained when acquiring the reference appearance information, and the quality of the apple in that case is suitable for determining the actual search solution. 5. By accumulating past data and analyzing the past data, the degree of association shown in FIG. 5 is created. Reference spectral information may be obtained, for example, through a measuring instrument, camera, or the like capable of analyzing spectral information.

For example, when evaluating the quality of actual apples in the past, it is assumed that reference spectrum information has a peak at OO nm in the near-infrared for certain reference appearance information. In such a case, if there are many cases in which the quality of apples is determined to be A, these are learned as a data set and defined in the form of the degree of association described above.

This analysis may be performed by artificial intelligence. In such a case, for example, when the reference appearance information P01 is the reference spectral information P20, the quality of the apple is analyzed from the past data. If there are many cases of apple quality A, the degree of association that links this apple quality to A is set higher, and if there are many cases of apple quality B and fewer cases of apple quality A , the degree of association in which the quality of apples leads to B is set high, and the degree of association in which the quality of apples leads to A is set to be low. For example, in the example of the intermediate node 61a, the outputs of apple quality A and apple quality B are linked. The degree of association of w14 which is connected is set to 2 points.

Further, the degree of association shown in FIG. 5 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence.

In the example of the degree of association shown in FIG. 5, the node 61b is a node of the combination of the spectral information for reference P18 with the appearance information for reference P01. degree is w16. The node 61c is a node of a combination of the reference spectral information P19 and P21 with respect to the reference appearance information P02. .

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data, the aforementioned learned data will be used when actually searching for the quality of apples. In such a case, the appearance information and spectrum information of the apple whose quality is to be determined are actually acquired. Here, the spectral information is newly acquired when actually estimating the quality of apples, and the acquisition method is the same as that for the reference spectral information described above.

Based on the appearance information newly acquired in this way and the spectral information, the optimal apple quality is searched for. In such a case, reference is made to the degrees of association shown in FIG. 5 (Table 1) that have been acquired in advance. For example, when the newly acquired appearance information is the same as or similar to P02 and the spectrum information is the same or similar to P21, the node 61d is associated via the degree of association. , and this node 61d is associated with apple quality C at w19 and apple quality D at w20. In such a case, the apple quality C with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality D, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

In FIG. 6, in addition to the above-mentioned reference appearance information, three or more levels of association are set between the combination of reference scent information instead of the above-mentioned reference variety information, and the quality of apples for the combination. shows an example.

This reference scent information, which is added as an explanatory variable instead of the reference position information, is all information related to the scent of apples. The reference scent information may be obtained from the apple through a smell sensor, a scent sensor, or the like. It is known that this apple scent has a great influence on the taste of apples. If the apple is rotten, it will have a foul odor, and naturally the taste will be deteriorated, and the quality will be reduced.

In the example of FIG. 6, it is assumed that the input data are, for example, reference appearance information P01 to P03 and reference scent information P18 to P21. An intermediate node shown in FIG. 6 is obtained by combining reference appearance information as input data with reference scent information. Each intermediate node is also connected to an output. In this output, the apple quality is displayed as the output solution.

Each combination (intermediate node) of reference appearance information and reference scent information is associated with each other through three or more levels of association with the quality of apples as the output solution. The appearance information for reference and the fragrance information for reference are arranged on the left side through this degree of association, and the quality of apples is arranged on the right side through the degree of association. The degree of association indicates the degree of high association between the reference appearance information and the reference fragrance information arranged on the left side and the quality of the apple. In other words, the degree of association is an index indicating to which apple quality each of the reference appearance information and the reference aroma information is likely to be linked. It indicates accuracy in selecting the most probable apple quality.

The estimating device 2 acquires in advance the degrees of association w13 to w22 of three or more stages shown in FIG. In other words, the estimating device 2 determines which of the reference appearance information, the reference scent information obtained when acquiring the reference appearance information, and the quality of the apple in that case is suitable for determining the actual search solution. By accumulating past data and analyzing the past data, the degree of association shown in FIG. 6 is created.

For example, it is assumed that the scent information for reference is the odor index equivalent value 〇〇 for some appearance information for reference when evaluating the quality of an actual apple in the past. In such a case, if there are many cases in which the quality of apples is determined to be A, these are learned as a data set and defined in the form of the degree of association described above.

This analysis may be performed by artificial intelligence. In such a case, for example, when the reference appearance information P01 is the reference scent information P20, the quality of the apple is analyzed from the past data. If there are many cases of apple quality A, the degree of association that links this apple quality to A is set higher, and if there are many cases of apple quality B and fewer cases of apple quality A , the degree of association in which the quality of apples leads to B is set high, and the degree of association in which the quality of apples leads to A is set to be low. For example, in the example of the intermediate node 61a, the outputs of apple quality A and apple quality B are linked. The degree of association of w14 which is connected is set to 2 points.

Further, the degree of association shown in FIG. 6 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence.

In the example of the degree of association shown in FIG. 6, the node 61b is a node of the combination of the appearance information for reference P01 and the scent information for reference P18. degree is w16. The node 61c is a node of a combination of the reference scent information P19 and P21 with respect to the reference appearance information P02. .

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data, the aforementioned learned data will be used when actually searching for the quality of apples. In such a case, the appearance information and fragrance information of the apple whose quality is to be actually determined are acquired. Here, the scent information is newly acquired when actually estimating the quality of apples, and the acquisition method is the same as the above-described reference scent information.

Based on the appearance information and aroma information newly acquired in this way, the optimal apple quality is searched for. In such a case, reference is made to the degrees of association shown in FIG. 6 (Table 1) that have been acquired in advance. For example, when the newly acquired appearance information is the same as or similar to P02 and the scent information is the same or similar to P21, the node 61d is associated via the degree of association. , and this node 61d is associated with apple quality C at w19 and apple quality D at w20. In such a case, the apple quality C with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality D, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

In FIG. 7, in addition to the above-described reference appearance information, three or more levels of association are set between the combination of the reference weight information instead of the above-described reference variety information, and the apple quality for the combination. shows an example.

This reference weight information added as an explanatory variable instead of the reference position information is any information about the weight of the apple. The reference weight information may be obtained from Apple through a weight scale or the like. It is known that the weight of this apple greatly affects the quality of the apple. If the apples are too small, they will have less pulp and be considered of poor quality.

In the example of FIG. 7, it is assumed that the input data are, for example, reference appearance information P01 to P03 and reference weight information P18 to P21. The intermediate node shown in FIG. 7 is a combination of the reference appearance information as input data and the reference weight information. Each intermediate node is also connected to an output. In this output, the apple quality is displayed as the output solution.

Each combination (intermediate node) of the reference appearance information and the reference weight information is associated with the quality of the apple as the output solution through three or more degrees of association. Appearance information for reference and weight information for reference are arranged on the left side through this degree of association, and apple quality is arranged on the right side through the degree of association. The degree of relevance indicates the degree of high relevance to the quality of the apple with respect to the reference appearance information and the reference weight information arranged on the left side. In other words, the degree of association is an index indicating the high possibility that each piece of reference appearance information and reference weight information is associated with the quality of an apple. It indicates accuracy in selecting the most probable apple quality.

The estimating device 2 acquires in advance the degrees of association w13 to w22 of three or more stages shown in FIG. In other words, the estimating device 2 determines which of the reference appearance information, the reference weight information obtained when acquiring the reference appearance information, and the quality of the apple in that case is preferable when determining the actual search solution. By accumulating past data and analyzing the past data, the degree of association shown in FIG. 6 is created.

For example, assume that the weight information for reference is 00 grams for some appearance information for reference when evaluating the quality of an actual apple in the past. In such a case, if there are many cases in which the quality of apples is determined to be A, these are learned as a data set and defined in the form of the degree of association described above.

This analysis may be performed by artificial intelligence. In such a case, for example, in the case of the reference appearance information P01 and the reference weight information P20, the quality of the apple is analyzed from the past data. If there are many cases of apple quality A, the degree of association that links this apple quality to A is set higher, and if there are many cases of apple quality B and fewer cases of apple quality A , the degree of association in which the quality of apples leads to B is set high, and the degree of association in which the quality of apples leads to A is set to be low. For example, in the example of the intermediate node 61a, the outputs of apple quality A and apple quality B are linked. The degree of association of w14 which is connected is set to 2 points.

Also, the degree of association shown in FIG. 7 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence.

In the example of the degree of association shown in FIG. 7, the node 61b is a node of the combination of the appearance information for reference P01 and the weight information for reference P18. degree is w16. The node 61c is a node of a combination of the reference weight information P19 and P21 with respect to the reference appearance information P02. .

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data, the aforementioned learned data will be used when actually searching for the quality of apples. In such a case, the appearance information and weight information of the apple whose quality is to be determined are actually obtained. Here, the weight information is newly acquired when actually estimating the quality of the apple, but the acquisition method is the same as the reference weight information described above.

Based on the newly acquired appearance information and weight information in this way, the optimal apple quality is searched for. In such a case, reference is made to the degrees of association shown in FIG. 7 (Table 1) that have been acquired in advance. For example, when the newly acquired appearance information is the same as or similar to P02 and the weight information is the same or similar to P21, the node 61d is associated via the degree of association. , and this node 61d is associated with apple quality C at w19 and apple quality D at w20. In such a case, the apple quality C with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality D, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

In FIG. 8, in addition to the above-described reference appearance information, three or more levels of association are set between the combination of the reference cultivation history information instead of the above-described reference cultivar information, and the apple quality for the combination. example.

This reference cultivation history information, which is added as an explanatory variable instead of the reference position information, is all information relating to the cultivation history of apples. The reference cultivation history information includes all cultivation histories from the period before the apple tree bears fruit to the harvest. The cultivation history information includes the timing, frequency, and amount of fertilization, watering, and pesticide application to the apple trees. Humidity, weather (history of sunny, cloudy, rainy, etc., timing of typhoons, etc.) are also included. In addition, since the temperature difference in a certain time unit (several hours, day, week, month) is also a factor that affects the quality of apples, this may be included in the reference cultivation history information. The reference cultivation history information may also include the place of production, and the information of the place of production may include information such as altitude, soil components, water content in the soil, and the like. The age of the apple tree may also be included as reference cultivation history information. Further, as the reference cultivation history information, chronological changes in the size and shape of apples as they grow may be combined with chronological changes in temperature and temperature difference in the same chronological order. This cultivation process affects the quality of apples. The process of cultivation may be obtained by inputting through a user interface or a user terminal.

In the example of FIG. 8, it is assumed that the input data are, for example, reference appearance information P01 to P03 and reference cultivation history information P18 to P21. The intermediate node shown in FIG. 8 is a combination of the reference appearance information as input data and the reference cultivation history information. Each intermediate node is also connected to an output. In this output, the apple quality is displayed as the output solution.

Each combination (intermediate node) of the reference appearance information and the reference cultivation history information is associated with each other through three or more levels of association with the apple quality as the output solution. Appearance information for reference and cultivation history information for reference are arranged on the left side through the degree of association, and the quality of apples is arranged on the right side through the degree of association. The degree of association indicates the degree of high relevance to the quality of the apple with respect to the reference appearance information and the reference cultivation history information arranged on the left side. In other words, the degree of association is an index indicating to which apple quality each piece of reference appearance information and reference cultivation history information is likely to be linked. It indicates the accuracy in selecting the most probable apple quality from the information.

The estimating device 2 acquires in advance the degrees of association w13 to w22 of three or more stages shown in FIG. In other words, the estimating device 2 determines which of the reference appearance information, the reference cultivation history information obtained when acquiring the reference appearance information, and the quality of the apple in that case, when determining the actual search solution. Whether it was suitable or not, past data is accumulated, and the degree of association shown in FIG. 8 is created by analyzing the data.

For example, when evaluating the quality of actual apples in the past, for certain reference appearance information, the reference cultivation history information was 〇〇 for fertilization, 〇〇 for watering, and a typhoon. It is assumed that the date of execution is August 16th. In such a case, if there are many cases in which the quality of apples is determined to be A, these are learned as a data set and defined in the form of the degree of association described above.

This analysis may be performed by artificial intelligence. In such a case, for example, in the case of the reference appearance information P01 and the reference cultivation history information P20, the quality of the apple is analyzed from the past data. If there are many cases of apple quality A, the degree of association that links this apple quality to A is set higher, and if there are many cases of apple quality B and fewer cases of apple quality A , the degree of association in which the quality of apples leads to B is set high, and the degree of association in which the quality of apples leads to A is set to be low. For example, in the example of the intermediate node 61a, the outputs of apple quality A and apple quality B are linked. The degree of association of w14 which is connected is set to 2 points.

Further, the degree of association shown in FIG. 8 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence.

In the example of the degree of association shown in FIG. 8, the node 61b is a node of a combination of the appearance information for reference P01 and the cultivation history information for reference P18. The association degree is w16. The node 61c is a node of a combination of the reference cultivation history information P19 and P21 with respect to the reference appearance information P02. there is

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data, the above-described learned data will be used when actually searching for the quality of apples. In such a case, the appearance information and the cultivation history information of the apple whose quality is to be determined are actually acquired. Here, the cultivation history information is newly acquired when actually estimating the quality of apples, and the acquisition method is the same as that for the reference cultivation history information described above.

Based on the appearance information newly acquired in this way and the cultivation history information, the optimal apple quality is searched for. In such a case, reference is made to the degrees of association shown in FIG. 8 (Table 1) that have been acquired in advance. For example, when the newly acquired appearance information is the same as or similar to P02 and the cultivation history information is the same or similar to P21, the node 61d is associated via the degree of association. This node 61d is associated with apple quality C with w19 and apple quality D with degree of association w20. In such a case, the apple quality C with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality D, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

In the present invention, it is also possible to similarly learn three or more degrees of association between the reference sugar content information regarding the sugar content of apples acquired in the past and the quality of apples. Then, the sugar content information about the sugar content of the apple to be estimated may be newly acquired, and the quality of the apple may be estimated using the degree of association from the reference sugar content information corresponding to the sugar content information. The details of this estimation method are the same as described above. Incidentally, the reference sugar content information and the sugar content information may be measured by a general sugar content meter.

As a substitute for the reference sugar content information and the sugar content information, the reference acidity information and the acidity information may be used to search for a solution. Reference acidity information and acidity information are the concentrations of acids contained in apples. In addition to titratable acidity measured by the neutralization titration method, all types of acids including acids detected by neutralization titration alone are included. It may be represented by the total acidity of the whole.

Further, hardness information for reference and hardness information may be used instead of sugar content information for reference and sugar content information. The hardness information for reference and the hardness information are information relating to the hardness of the apple, and may be measured with a general hardness tester or evaluated by a texture test, for example.

When estimating the quality of apples before thinning, three or more degrees of association between the reference morphological information including the apples before thinning and the branches connected thereto and the quality of apples may be used. . The apples before thinning and the branches connected thereto are composed of image data including the apples before thinning and the branches connected thereto, or numerical data or charts showing the relationship between the apples before thinning and the branches. Thickness, angle, length, degree of branching, direction of branching, number of branches, number of apples growing on the branched branch, and branch of the apple concerned You may also include how to attach to and aspects.

This reference morphological information may include the relationship between a leaf growing on a branch and an apple. The relationship between the leaves growing on the branch and the apple is information such as the positional relationship between the leaf and the apple, the density of the apple with respect to the amount of leaves, the distance of each leaf from the apple, and the size of each leaf.

Even in such a case, new morphological information is acquired during solution search. The morphological information is information about the apple to be discriminated before fruit thinning and the branch connected thereto, and includes the relationship between the leaf growing on the branch and the apple if necessary. The method of searching for a solution by using the acquired morphological information and referring to the degree of association is the same as described above.

In such a case, three or more degrees of association may be formed in advance between the above-described reference morphological information and the pattern relating to the attachment of apples to branches or the pattern relating to the attachment of leaves to apples. The type here may be a type freely determined by the system side or the user side in advance. The types here are, for example, type P, which has only one apple at the tip of a thick branch and almost no leaves, type Q, which has a plurality of apples at the tip of a thin branch, and the thin branch Branches with many small leaves, type R has a thick branch that branches into two, and one of the branches has multiple apples. There are a lot of apples, but no apples, and so on. Such a pattern is determined in advance, and the reference form information is actually applied to the pattern. This fitting may be performed by an expert or the like. Then, each form information for reference and a degree of association are formed by learning a data set consisting of patterns. It goes without saying that this reference morphological information may include the relationship between the apple before fruit thinning and the leaves growing on the branches connected to it.

As a result, when new morphological information is input, it is possible to perform a solution search in the same manner as described above by referring to the degree of association.

Note that each type described above and the quality of the apple may be associated with each other. Since the quality of apples may change according to the type described above, the quality of apples is linked to the type on a one-to-one basis. Then, based on the above-described method, when a pattern as an output solution can be searched, the quality of the apple associated with the searched pattern may be estimated.

At this time, the reference morphological information of the entire tree including the apples before fruit thinning and the branches connected thereto may be learned. This is because the quality of apples may be governed by the shape of the whole tree, rather than by the branches or apples that are part of the tree. In such a case, it goes without saying that the morphological information of the entire tree on which the target apple to be estimated grows is similarly acquired as the morphological information.

In the above-mentioned degree of relevance, the degree of relevance is expressed by a 10-level evaluation, but it is not limited to this, and it may be expressed by a degree of relevance of 3 or more stages. For example, 100 steps or 1000 steps may be used. On the other hand, this degree of association does not include those expressed in two stages, that is, whether or not they are associated with each other, either 1 or 0.

According to the present invention configured as described above, anyone can easily search for the quality of apples without special skills or experience. Further, according to the present invention, it is possible to judge the search solution with a higher degree of accuracy than a human being does. Furthermore, by configuring the degree of association described above with artificial intelligence (neural network, etc.) and making it learn, it is possible to further improve the discrimination accuracy.

Since the above-described input data and output data may not be exactly the same in many cases during the learning process, information obtained by classifying these input data and output data according to type may be used. That is, the information P01, P02, . . . P15, 16, . A data set may be created between the data and the output data for learning.

In addition, in the degree of association described above, in addition to the reference appearance information, any combination of reference cultivar information, reference spectrum information, reference scent information, reference weight information, reference cultivation history information, and reference sugar content information Although the description has been given taking the example of the case where it is configured with, the present invention is not limited to this. That is, the degree of association is a combination of two or more of the reference cultivar information, the reference spectrum information, the reference fragrance information, the reference weight information, the reference cultivation history information, and the reference sugar content information in addition to the reference appearance information. may be composed of In addition to the reference appearance information, the degree of association is one or more of reference variety information, reference spectrum information, reference scent information, reference weight information, reference cultivation history information, and reference sugar content information, Other factors may be added to this combination to form the degree of relevance.

In either case, data is input in accordance with the reference information of the degree of association, and the quality of the apple is determined using the degree of association.

Moreover, according to the present invention, it is characterized in that the optimum solution search is performed through the degree of association set to three or more stages. The degree of association can be described by a numerical value of, for example, 0 to 100% in addition to the 10 stages described above, but is not limited to this, and can be described by a numerical value of 3 or more stages. may be configured.

By discriminating the most probable apple quality based on the degree of association represented by such three or more levels of numerical values, under circumstances where there are multiple candidates for the possibility of search solutions, the degree of association with high It is also possible to search and display them in order. If it is possible to display to the user in descending order of degree of association in this way, it is also possible to preferentially display more probable search solutions.

In addition to this, according to the present invention, it is possible to make a judgment without overlooking even a very low output discrimination result such as a correlation degree of 1%. Remind the user that even discrimination results with an extremely low degree of association are connected as slight signs, and that once in dozens or hundreds of times, the discrimination results may be useful. be able to.

Furthermore, according to the present invention, there is an advantage that a search policy can be determined by setting thresholds by performing a search based on three or more degrees of association. If the threshold value is lowered, even if the degree of association is 1%, it can be picked up without omission. be. On the other hand, if the threshold is set high, there is a high possibility that the optimal search solution can be detected with a high probability. Sometimes the solution is overlooked. It is possible to decide which one to place emphasis on based on the way of thinking of the user side and the system side, and it is possible to increase the degree of freedom in selecting such points to place emphasis on.

Furthermore, in the present invention, the degree of association described above may be updated. This update may reflect information provided via a public communication network such as the Internet, for example. In addition, external appearance information for reference, cultivar information for reference, spectral information for reference, scent information for reference, weight information for reference, cultivation history information for reference, and sugar content information for reference are acquired, and knowledge about apple quality and improvement measures for these , information, and data are acquired, the degree of association is increased or decreased accordingly.

In other words, this update corresponds to learning in terms of artificial intelligence. Since new data is acquired and reflected in the learned data, it can be said that it is a learning act.

In addition to updating the degree of association based on information that can be obtained from the public communication network, the system side or the user side based on the contents of research data, papers, conference presentations, newspaper articles, books, etc. by experts It may be updated manually or automatically. Artificial intelligence may be used in these update processes.

Also, the process of initially building a trained model and the above-described updating may use not only supervised learning but also unsupervised learning, deep learning, reinforcement learning, and the like. In the case of unsupervised learning, instead of loading and learning datasets of input and output data, learning is performed by loading information corresponding to the input data, and from there self-forming the degree of association related to the output data. You can let it run.

Second Embodiment In the example of FIG. 9, the input data are assumed to be reference cultivation history information P01 to P03, for example. Such reference cultivation history information P01 to P03 as input data are linked to apple quality as output. In this output, the apple quality is displayed as the output solution.

The reference cultivation history information is associated with the quality A to D of apples as the output solution through three or more degrees of association. Cultivation history information for reference is arranged on the left side through this degree of association, and the quality of each apple is arranged on the right side through the degree of association. The degree of relevance indicates the degree of relevance to the quality of which apple with respect to the reference cultivation history information arranged on the left side. The estimating device 2 acquires in advance the degrees of association w13 to w19 of three or more stages shown in FIG. In other words, the estimating device 2 accumulates a past data set indicating which of the reference cultivation history information and the quality of apples in that case was adopted and evaluated in order to determine the actual search solution. is analyzed to create the degrees of association shown in FIG. This analysis may be performed by artificial intelligence. That is, it may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficients for the outputs of the nodes of this neural network correspond to the degrees of association described above. Moreover, it is not limited to a neural network, and may be composed of all decision-making factors that constitute artificial intelligence. Such a degree of association becomes learned data in terms of artificial intelligence.

At the time of actual quality discrimination, new cultivation history information is acquired for the apple to be discriminated. The cultivation history information to be newly acquired is input by the information acquisition unit 9 described above.

Based on the cultivation history information newly acquired in this way, the quality of the apple is determined. In such a case, reference is made to the degrees of association shown in FIG. 9 (Table 1) that have been acquired in advance. For example, when the newly acquired cultivation history information is the same as or similar to P02, the apple quality B is associated with w15 and the apple quality C is associated with the association degree w16. . In such a case, the apple quality B with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality C, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

In this way, it is possible to search for the most suitable apple quality from the newly acquired cultivation history information and display it to the user.

FIG. 10 shows an example in which three or more degrees of association are set between a combination of reference cultivar information in addition to the above-described reference cultivation history information and apple quality for the combination.

In the example of FIG. 10, the input data are, for example, reference cultivation history information P01-P03 and reference variety information P18-21. An intermediate node shown in FIG. 10 is obtained by combining reference cultivar information with reference cultivation history information as such input data. Each intermediate node is also connected to an output. In this output, the apple quality is displayed as the output solution.

Each combination (intermediate node) of the reference cultivation history information and the reference cultivar information is associated with the apple quality as the output solution through three or more degrees of association. The cultivation history information for reference and the reference cultivar information are arranged on the left side through this degree of association, and the quality of apples is arranged on the right side through the degree of association. The degree of relevance indicates the degree of relevance to the quality of apples with respect to the reference cultivation history information and the reference cultivar information arranged on the left side. In other words, the degree of association is an index indicating to which apple quality each of the reference cultivation history information and the reference cultivar information is likely to be linked. It indicates the accuracy in selecting the most probable apple quality from the information.

The estimating device 2 acquires in advance the degrees of association w13 to w22 of three or more stages shown in FIG. In other words, the estimating device 2 determines which of the reference cultivation history information, the reference variety information obtained when acquiring the reference cultivation history information, and the quality of the apple in that case, when determining the actual search solution. was suitable, accumulate past data, analyze the past data, and build up the degree of association shown in FIG.

This analysis may be performed by artificial intelligence. In such a case, for example, when the reference cultivation history information P01 is the reference variety information P20, the quality of the apple is analyzed from the past data. If there are many cases of apple quality A, the degree of association that links this apple quality to A is set higher, and if there are many cases of apple quality B and fewer cases of apple quality A , the degree of association in which the quality of apples leads to B is set high, and the degree of association in which the quality of apples leads to A is set to be low. For example, in the example of the intermediate node 61a, the outputs of apple quality A and apple quality B are linked. The degree of association of w14 which is connected is set to 2 points.

In the example of the degree of association shown in FIG. 10, the node 61b is a node of a combination of the reference cultivation history information P01 and the reference cultivar information P18. The association degree is w16. The node 61c is a node of a combination of the reference cultivar information P19 and P21 with respect to the reference cultivation history information P02. there is

Such a degree of association becomes learned data in terms of artificial intelligence. After creating such learned data, the aforementioned learned data will be used when actually searching for the quality of apples. In such a case, the cultivation history information and variety information of the apple whose quality is to be actually determined are obtained. Here, the cultivar information is newly acquired when actually estimating the quality of apples, and the acquisition method is the same as the above-described reference cultivar information.

Based on the cultivation history information newly acquired in this way and the variety information, the optimal apple quality is searched for. In such a case, reference is made to the degrees of association shown in FIG. 10 (Table 1) that have been acquired in advance. For example, when the newly acquired cultivation history information is the same as or similar to P02, and the variety information is the same or similar to P21, the node 61d is associated via the degree of association. This node 61d is associated with apple quality C with w19 and apple quality D with degree of association w20. In such a case, the apple quality C with the highest degree of association is selected as the optimum solution. However, it is not essential to select the one with the highest degree of association as the optimum solution, and the apple quality D, which has a low degree of association but is recognized for its association itself, may be selected as the optimum solution. In addition, it is of course possible to select an output solution that is not connected by an arrow, and any other priority may be used as long as it is based on the degree of association.

The degree of association shown in FIG. 10 described above may be composed of a combination of any one of reference spectrum information, reference scent information, and reference weight information instead of the reference product type information. In such a case, as input data, in addition to cultivation history information, spectrum information, scent information, and weight information are input, and solution search is performed based on these based on the method described above.

In addition to the reference cultivation history information, the degree of association may be a combination of two or more of reference variety information, reference spectrum information, reference scent information, and reference weight information. In addition to the reference cultivation history information, the degree of association is obtained by adding one or more of the reference cultivar information, the reference spectrum information, the reference scent information, and the reference weight information, and other factors to this combination. degrees may be formed.

Further, both the first embodiment and the second embodiment are not limited to the embodiments described above. For example, as shown in FIG. may be used. In such a case, the solution search is performed based on three or more degrees of association between the reference information corresponding to the newly acquired information and the quality of the apple. The basic reference information includes all of the above-mentioned reference information (reference appearance information, reference cultivation history information, reference cultivar information, reference spectrum information, reference scent information, reference weight information, reference sugar content information, etc.) can be applied.

In these cases, similarly, when information corresponding to reference information used as learning data is input, solution search is performed based on the above-described method.

The search solution obtained through the degree of association may be further modified or weighted based on other reference information.

The other reference information referred to here corresponds to any reference information other than the basic reference information when any of the reference information described above is used as the basic reference information.

For example, it is assumed that in certain reference cultivation history information P14, which is one of the other reference information, there were many circumstances in which quality B of apples was determined before. When the cultivation history information corresponding to the reference cultivation history information P14 is newly acquired, the search solution B as the quality of the apple is weighted, in other words, the search solution of the quality of the apple. It is set in advance so as to perform processing to connect to B.

For example, the other reference information G is an analysis result that suggests the search solution C as the quality of apples, and the reference information F is the analysis that suggests the search solution D as the quality of apples. Shall be the result. After the setting with the reference information in this way, when the actually obtained information is the same as or similar to the reference information G, processing is performed to increase the weighting of the quality C of the apple. On the other hand, when the actually acquired information is the same as or similar to the reference information F, processing is performed to increase the weight of the quality D of the apple. In other words, the degree of association itself leading to the quality of apples may be controlled based on the reference information FH. Alternatively, after the quality of the apple is determined only by the degree of association described above, the obtained search solution may be corrected based on the reference information FH. In the latter case, how and with what weight the quality of the apple as the search solution is modified based on the reference information FH reflects what is designed on the system side each time. In addition, as other reference information used here, sunshine information related to the sunshine applied to apples being grown may be used. The sunshine information is information about from which direction the sunlight hits the apple tree, the time period for each direction of the sunshine, and the sunshine duration. A search solution may be obtained in the same manner as described above based on the obtained sunshine information.

The reference information is not limited to any one type, and the solution search may be performed based on two or more types of reference information. In this case, similarly, the discriminant pattern as a search solution obtained through the degree of association may be corrected to a higher value for a case that leads to the apple quality suggested by the reference information.

Similarly, as shown in FIG. 12, even in the case of forming the degree of association with the quality of apples for a combination of reference information that is the keynote and other reference information, the reference information that is the keynote is , any reference information in the first and second embodiments (in addition to reference appearance information and reference cultivation history information, reference variety information, reference spectrum information, reference fragrance information, reference weight information, reference sugar content information, etc.) can also be applied. Other reference information includes any reference information in the first and second embodiments other than the basic reference information.

At this time, if the basic reference information is the reference appearance information, the other reference information includes any reference information in the first embodiment and the second embodiment. In such a case, if the basic reference information and other reference information are reference form information and reference appearance information, the reference appearance information related to the appearance of the apple when acquiring the reference form information and The degree of association is composed of combinations having

In such a case as well, the quality of the apple can be estimated by searching for the solution in the same way. At this time, as shown in FIG. 11 described above, the quality of the apple is corrected through other reference information (reference information F, G, H, etc.) for the search solution obtained through the degree of association. You may do so.

At this time, the degree of association may be learned by combining not only one piece of other reference information but also two or more pieces of information.

Further, as shown in FIG. 13, the degree of association may be formed between only the reference information that is the keynote and the quality of the apple. This basic reference information is any reference information in the first and second embodiments (reference appearance information, reference cultivation history information, reference cultivar information, reference spectrum information, reference scent information, weight information for reference, sugar content information for reference, etc.) are also applicable. Description of the solution search method in FIG. 13 will be omitted by citing the description of FIG.

Third Embodiment In the third embodiment, thinning information is searched as a search solution on the output side. This fruit thinning information is information about whether the apple is actually thinned from the central flower or from which lateral flower when the apple fruit is actually harvested from the apple tree. Apples usually bloom in groups of 4 to 6 flowers. When flowers actually bloom, the central flower blooms first. Next, multiple lateral flowers around the central flower bloom later. In general, the central flower is often the most nutritious, so apples with central flowers are often of better taste and quality than apples with lateral flowers. For this reason, usually, by thinning out the side flowers or the apple fruits that are the side flowers, the apple that is the central flower is grown large. However, when it is difficult to grow an apple on the central flower for some reason, or when it is intentionally not possible to grow an apple on the central flower, apples that are lateral flowers may be grown. .

In this way, it is possible to estimate the quality of an apple by determining whether the apple consists of central flowers or lateral flowers. By connecting the apple that is the central flower and its quality, and the apple that is the lateral flower and its quality, it is possible to determine whether the apple has grown from either the central flower or the lateral flower. It is also possible to estimate the assigned quality.

In addition, as the dripping fruit information, it is not determined only by the binary value of whether the apple is composed of the central flower or the apple composed of the side flowers, but it is related to what kind of side flower even if it is the same side flower. Information may also be included in this.

In order to actually perform such an estimation, as shown in FIG. 14, three or more degrees of association between the reference appearance information and the thinning information regarding whether the apple was thinned from the central flower or from which side flower. take advantage of In this case, the reference appearance information to be learned in advance when searching for the thinning information as a search solution includes the appearance shape of the apple, the direction of the core of the apple to be estimated, the striped pattern on the surface of the apple, and the shape of the apple before thinning. It consists of apples and branches connected to them.

This apple appearance shape is composed of the shape of the outer shell of the apple. Whether the apple is in a state close to a circle, a shape close to a column, or a state close to flat, or even if the same apple before thinning and branches connected to it are close to a circle, the curvature of the outer shell may be slightly different. There are also cases where flat surfaces are formed here and there instead of arcs. In this way, the external shape of the apple is obtained as numerical data or an image.

The direction of the apple core is information indicating in what direction the linear core protruding from the top of the apple faces. The orientation of this apple core consists of measured numerical data or an image.

The striped pattern on the surface of apples varies depending on the variety, but there is a relationship between the data on which direction the striped pattern is oriented and whether the apple is the central flower or from which side flower. I found something and included it in one of the appearance information. The striped pattern on the surface of this apple is composed of image data.

The apples before thinning and the branches connected thereto are composed of image data including the apples before thinning and the branches connected thereto, or numerical data or charts showing the relationship between the apples before thinning and the branches. Thickness, angle, length, degree of branching, direction of branching, number of branches, number of apples growing on the branched branch, and branch of the apple concerned You may also include how to attach to and aspects.

A data set consisting of such reference appearance information and thinning information is learned in advance. Then, new appearance information is acquired. This acquired appearance information is made to correspond to reference appearance information. That is, any one of the appearance shape of the apple, the direction of the core of the apple to be estimated, the striped pattern on the surface of the apple, the apple before thinning and the branches connected thereto, which constitute the appearance information for reference, is associated with the appearance information. The external shape of the apple as the target, the direction of the core of the apple to be estimated, the striped pattern on the surface of the apple, the apple before fruit thinning, and the branches connected thereto are acquired.

Then, by inputting the acquired appearance information, the thinning information is searched through the corresponding reference appearance information. The details of this search are omitted below by citing the descriptions of the first and second embodiments.

In the third embodiment, it may end by displaying the thinning information that has been searched, but it is not limited to this, and the quality of the apple linked in advance to the thinning information is displayed. may

Further, in the third embodiment, as shown in FIG. 15, the degree of association with thinning information may be formed for a combination of reference information serving as a keynote and other reference information. In such a case, the basic reference information includes the appearance shape of the apple that constitutes the reference appearance information, the direction of the core of the apple to be estimated, the striped pattern on the surface of the apple, the apple before thinning and the branches connected to it, Other reference information includes the appearance shape of the apple, the direction of the core of the apple to be estimated, the striped pattern on the surface of the apple, the apple before fruit thinning and the branches leading to it. It is composed of any information other than the reference information. For example, if the basic reference information is the external shape of an apple, the other reference information is the direction of the core of the apple, the striped pattern on the surface of the apple, the apple before thinning and the branches leading to it, Consists of either

Description of the solution search method for the degree of association shown in FIG. 15 will be omitted by citing the descriptions of the above-described first and second embodiments.

That is, in the example shown in FIG. 15, the external shape of the apple to be estimated, the direction of the core of the apple to be estimated, the apple to be estimated before thinning and the branches connected thereto, and the striped pattern on the surface of the apple to be estimated. Appearance information of any two or more of them is acquired. Two or more reference appearance information prepared in advance among the appearance shape of the apple, the direction of the core of the apple, the apple before fruit thinning and the branches connected thereto, and the striped pattern on the surface of the apple, according to the appearance information; The thinning information is searched based on the appearance information for reference corresponding to the obtained appearance information, using three or more degrees of association with the thinning information about whether the apple has been thinned from the central flower or from which side flower.

Note that, in the third embodiment, as the basic reference information, the appearance shape of the apple that constitutes the reference appearance information, the direction of the core of the apple to be estimated, the striped pattern on the surface of the apple, and the apple before thinning. and a branch connected thereto, any reference information in the first embodiment and the second embodiment may be learned as other reference information. At this time, if the basic reference information is the above-described reference appearance information, the other reference information includes any reference information in the first embodiment and the second embodiment.

In such a case as well, the quality of the apple can be estimated by searching for the solution in the same way. At this time, as shown in FIG. 11 described above, further other reference information (reference information F, G, H, etc.) to modify the apple quality.

At this time, the degree of association may be learned by combining not only one piece of other reference information but also two or more pieces of information.

That is, appearance information of any two or more of the external shape of the estimated target apple, the orientation of the estimated target apple core, the estimated target apple before fruit thinning and the branches connected thereto, and the striped pattern of the estimated target apple surface. to get In such a case, reference appearance information relating to the appearance shape of the apple, the direction of the core of the apple, the apple before fruit thinning and the branches connected thereto, and the striped pattern on the surface of the apple, according to any one of the above appearance information, and the apple Three or more degrees of association with fruit thinning information about whether the fruit is thinned from the central flower or from which side flower are prepared in advance. As described above, when appearance information is received as input data, thinning information is searched for by referring to the degree of association prepared in advance. In addition to this, the thinning information is searched based on other acquired appearance information. For example, when the reference appearance information is the direction of the apple core, the direction of the apple core is obtained as the appearance information, and the degree of association is used to search for the thinning information. At this time, the thinning information may be searched based on other appearance information (one or more of the shape of the apple, the apple before thinning and the branches connected thereto, and the striped pattern on the surface of the apple). .

In the present invention, the central and lateral flowers of the apple tree may also be plucked at a stage where the central and lateral flowers are in bloom, or in a state before and after that, and the apple tree has not yet grown apple fruit. You may make it suggest the flower which should be done. The flower to be picked here is information about the flower to be picked.

In such a case, as shown in FIG. 16, three or more degrees of association are used between the reference image information captured in the past so as to include the central flower and the side flowers, and the flower to be picked. The flowers to be pruned are when the central flower and side flowers are in bloom, or before and after the apple tree. You can ask them in an interview. At this time, reference image information is obtained by capturing images of the central flower and lateral flower of the apple tree that have been identified.

A data set of such reference image information and flowers to be picked is learned. Then, when it is desired to newly propose flowers to be plucked with respect to the central and lateral flowers of the apple tree, images of the central and lateral flowers of the apple tree to be proposed are picked up and image information is obtained. get Based on reference image information corresponding to this image information, a flower to be picked is proposed with reference to the degree of association. The search method for the search solution is the same as described above.

At this time, when the information acquisition unit 9 or the information acquisition unit 9 and the estimation device are configured by a glasses-type terminal or a head-mounted display (HMD), an image is captured through the HMD so as to include the central flower and the side flower of the proposal target. Image information may be obtained by Then, the flower to be picked as the obtained search solution may be displayed on the display of the HMD. In such a case, so-called AR (Augmented Reality) technology may be used to display information on proposed flowers to be picked on a transparent display. Thereby, in the image visually recognized on the screen, the information regarding the flower to be picked can be displayed in a transparent state, and the workability of the user can be improved.

In the second embodiment and the third embodiment, as in the first embodiment, as a search solution, instead of the quality of apples, diseases occurring in apples may be applied. This makes it possible to estimate diseases occurring in apples by a similar procedure.

Also in the first to third embodiments, as a search solution, apple harvest time information may be applied as a substitute for apple quality. As a result, apple harvest time information can be estimated in a similar manner.

Fourth Embodiment In the fourth embodiment, instead of the quality of apples, the method of cultivating apples is obtained as a search solution.

FIG. 17 shows an example of the degree of association with the cultivation method as the search solution. In this example, the input is morphological information for reference, and the output is the cultivation method of apples. Since the fourth embodiment proposes an optimum cultivation method for apples that are currently being grown, the reference information corresponding thereto includes morphological information for reference, cultivation history information for reference, and appearance for reference. Although it is desirable to apply information or the like, it is not limited to these, and any reference information described in the above first to third embodiments may be applied.

The cultivation method here is information on the optimum cultivation method for apples currently being grown. For example, "spray XX fertilizer twice a week, spray pesticide △△ once a month" or "spray water twice a day, and remove flowers at the timing of XX". It may contain content close to advice on methods. In addition, the cultivation method includes the number and timing of additional fertilization, the number and timing of disinfection, the timing of pruning and pollination, the timing of fruit thinning, the timing and method of bagging and removal, and the timing and method of placing stakes. The timing and method of picking, ball spinning, etc. may be specified. In particular, pruning reflects the opinions of experts on which branches to cut and how to arrange the branches so that sunlight can reach the apples inside the tree, and then converts it into information and data. back.

Then, when information corresponding to the reference information is newly input, a cultivation method as an output solution is searched for through the degree of association described above, and a display or the like for proposing it to the user is performed. At this time, as shown in FIG. 12, a plurality of pieces of reference information may be input to search for a solution. Alternatively, as shown in FIG. A solution search may be performed.

Thus, in the process of growing apples, by sequentially acquiring morphological information for reference, cultivation history information for reference, appearance information for reference, etc., morphology information, cultivation history information, appearance information, etc. It becomes possible to propose a more suitable cultivation method to the user.

Fifth Embodiment In the fifth embodiment, the estimating device 2, or the estimating device 2 and the information acquisition unit 9, among wearable terminals, in particular, glasses-type terminals and head-mounted displays (HMDs) are used. This HMD is mounted integrally or partially on the user's head or eyeglasses, and utilizes technology such as augmented reality (AR) or mixed reality (MR), based on various image information acquired. and a display unit that displays the information generated by the process in a transparent state. The user can visually recognize and understand the information to be displayed through the display unit that transparently displays the information on the HMD. As a result, the user can check the information generated based on the various types of acquired video information and various contents together while looking at the situation in front of the user.

Therefore, in the present invention, for example, appearance information and form information are acquired via the information acquisition unit 9 mounted on this HMD. Then, the estimating device 2 mounted in the HMD searches for a solution, and the obtained search solution (the quality of the apple, its selling price, etc.) may be displayed in a transparent state through the display unit.

Appearance information for reference and morphological information for reference to be learned as learning data in the fifth embodiment may be actually captured by an eyeglass-type terminal such as an HMD. The image is not limited, and may be an image captured by a normal digital camera, smartphone, or the like.

In addition, in actually obtaining the appearance information for reference and the morphological information for reference, it is possible to associate which part of the apple is imaged.

For example, to obtain reference appearance information, detect what part of the apple a skilled and experienced farmer who evaluates the quality of the apple is looking at. It is possible to detect whether a veteran farmer is visually recognizing mainly the bottom part of the apple, recognizing mainly the core part of the apple, or recognizing mainly the side part of the apple. .

For this detection, for example, a veteran farmer is made to wear a spectacles-type terminal and actually pick up an apple to detect it. The image is continuously captured in the direction the farmer is visually recognizing. Then, by analyzing the recorded image after the fact or reproducing the image, it is possible for a veteran farmer to actually evaluate the quality of apples through the appearance information and morphological information of apples. It is possible to detect which part of the is being visually recognized.

In such a case, as shown in FIG. 18, when moving images captured by the glasses-type terminal are arranged in time series, an image of the bottom of the apple (P1), an image of the core of the apple (P2), and an image of the core (P2). Assume that the image is an enlarged image (P3) and a side image (P4). Information on the body part to be imaged may be detected from the images obtained in time series in this way. The imaging target part information here is information regarding which part of the apple is captured by the image captured by the glasses-type terminal. As shown in FIG. 18, this imaging target region information may be composed of names of regions actually being imaged, such as the bottom, core, side, etc., or symbols for specifying the regions. It may be represented by a numerical value, a number, or the like. The imaging target site information may also include, for example, whether the imaging is an enlarged image or a reduced image, and information such as the imaging direction and angle of view at the time of imaging.

The acquisition of the imaging target region information may be performed manually by a human identifying the imaging target region each time, or may be obtained by using a well-known image analysis technique for the acquired image. Acquisition of this imaging target region information may be determined based on previously learned feature amounts. For example, images of each part such as the bottom, core, and sides of an apple may be identified by extracting them through artificial intelligence using deep learning technology. In such a case, a machine learning model is used in which the parts of the apple included in the reference appearance information and the imaging target part information are used as training data, the input is the reference appearance information, and the imaging target part information is the output. . Then, based on the appearance information for reference newly imaged via the user terminal, the imaging target part information is acquired. In addition to this, acquisition of imaging target part information is performed by using the direction of the line of sight detected using the eye-tracking function installed in the HMD and glasses-type terminal, and the direction of the head detected using the acceleration sensor and the gyro sensor. The body part information to be imaged may be acquired through the orientation of the part, the movement of the user's hand using the operation device or the hand tracking function, and the like.

By recording the imaging target part information thus obtained in association with each of the images P1 to P4, etc., it is possible to set which part of the apple each of the images P1 to P4 is an image of. can be obtained at Supposing that images P1 to P4 constituting such reference appearance information and imaging target region information are obtained from a veteran farmer, the veteran farmer can determine which region to use when actually inspecting apples. It is also possible to acquire whether or not the images are being checked in order, whether or not the images are enlarged images, and the shooting environment such as the shooting angle. As for the morphological information for reference, it is possible to similarly acquire the imaging target region information and record it in association with it.

In the fifth embodiment, it is possible to improve the convenience during the actual solution search by utilizing the imaging target region information. When a user wearing an HMD implements the present invention after building the above-described learning data in advance, the search for search solutions (quality, sales price, etc.) is described in the above-described first to fourth embodiments. based on the method used. At this time, the imaging target part information may be obtained in the same way in the process of obtaining the imaged appearance information and morphological information of the apple that the user wearing the HMD is visually recognizing. As a method of acquiring this imaging target site information from appearance information and morphological information, determination may be made based on the feature amount learned as described above. You may make it discriminate|determine by extracting the image of each part of apples, etc. through artificial intelligence using a deep-learning technique. In such a case, a machine learning model is used in which the part of the apple included in the appearance information and the imaging target part information are used as teacher data, the input is the appearance information, and the imaging target part information is the output. Then, the imaging target part information is acquired based on the appearance information newly imaged via the user terminal. The imaging target site information can be similarly obtained from the morphological information.

Hereinafter, the imaging target region information acquired from information including appearance information, morphological information, etc., is also referred to as first imaging target region information, and is obtained from reference information including reference appearance information, reference morphology information, etc. The imaging target region information is also referred to as second imaging target region information.

Acquiring such first imaging target region information each time in the process of acquiring appearance information and morphological information provides the excellent effects described below. It is possible to confirm each time whether the first imaging target region information in the imaged appearance information matches the second imaging target region information in the reference appearance information. When the first imaging target region information in the imaged appearance information and the second imaging target region information in the reference appearance information do not match, a warning can be displayed to the user via the display unit of the HMD.

For example, if the first imaging target part information in the imaged appearance information is "side" and the second imaging target part information in the reference appearance information for corresponding to this is "core", the HMD is worn. This means that the part of the apple for which the user photographs the appearance information is the wrong part. In such a case, it is possible to prompt the user to align the imaging target with the correct part by calling attention as described above. Further, while the second imaging target part information linked to the reference appearance information is for visually magnifying the bottom part, the first imaging target part information linked to the appearance information is the same bottom part. However, if the image is not enlarged and viewed, it is possible to prompt the user to enlarge and view the image in the same manner.

In this way, the degree of matching between the first imaging target region information of the captured appearance information and the second imaging target region information of the reference appearance information, or the imaging target region information of the captured morphology information and the reference morphology information Via the degree of matching with the imaging target site information, it is possible to make various suggestions to the user wearing the HMD about the imaging method of the actual appearance information and morphological information, or prompt various corrections. . At this time, the above-described AR or MR may be realized in which such suggestions and promotion of correction are displayed in a transmissive state through the display unit of the HMD or glasses-type terminal. It should be noted that any suggestion about the imaging method may be displayed as long as it is based on the first imaging target region information and the second imaging target region information.

In the example of FIG. 18 described above, the description has been given by taking as an example the case of sequentially switching the imaging target for a plurality of locations for the apple, but the present invention is not limited to this, and the appearance information for reference or the like can be obtained for a single apple. Similarly, by acquiring and linking the imaging target region information even in the case of the other, it is possible to perform the above-described guidance to the user who actually acquires the appearance information and the like.

In the present invention, as shown in FIG. 19, three or more degrees of association between the appearance information for reference or the morphological information for reference and the quality of the apple may be obtained in advance for each part of the apple. . In such a case, the degree of association between the reference appearance information or the reference form information and the quality of the apple is independently learned and stored for each part of the apple such as the side, the bottom, and the core. back.

Then, the degrees of association of the parts of the apple corresponding to the acquired first imaging target part information are read out, and solution search is performed in the same manner as described above. If the first imaging target part information is the bottom of an apple, an evaluation model consisting of the degree of association for the bottom is selected and quality estimation is performed. As a result, an evaluation model specialized for each part of the apple can be selected, and quality estimation can be performed with higher accuracy.

In addition, as shown in FIG. 20, the present invention may construct the degree of association for each part of the apple as described above, and classify the degree of association for each reference cultivation history information. In such a case, for example, as shown in FIG. The degree of association is formed for each part of the apple, such as for the side, for the bottom, and for the core. The degree of association for each part, which is classified for each reference cultivation history information, is prepared. Then, when a new solution search is performed, the above-described imaging target part information is acquired, and cultivation history information is acquired. Then, an evaluation model classified for each part of the apple in the cultivation history corresponding to the cultivation history information is selected, and solution search is similarly performed.

Sixth Embodiment Next, an example of an apple quality estimation model construction system and an apple quality estimation system in a sixth embodiment will be described. The difference between the above-described embodiment and the sixth embodiment is the use of image pickup target product information. In addition, description is abbreviate|omitted about the content similar to embodiment mentioned above.

The imaging target variety information indicates information about apple varieties. The information about the variety to be photographed can be obtained by using well-known image analysis technology based on the appearance information and morphological information obtained by photographing the appearance of the apple. good.

For example, acquisition of photographing target variety information may be determined based on previously learned feature amounts. For example, based on the appearance of an apple, deep learning techniques may be used to estimate quality through artificial intelligence. In such a case, machine learning is performed by using appearance information for reference (or morphological information for reference, the same shall apply hereinafter) and information on the cultivar to be photographed as training data, appearance information for reference as input, and cultivar information to be photographed as output. make use of the model. Then, based on appearance information (or morphological information, the same shall apply hereinafter) that has been newly imaged via a user terminal or the like, photographing target variety information corresponding to the appearance information is acquired. The details of appearance information and morphological information and the acquisition method thereof are the same as in the above-described embodiment, and thus descriptions thereof are omitted.

The imaging target variety information may be estimated, for example, based on morphological information including the appearance of branches leading to apples. In this case, the apple variety can be estimated based on the characteristics of the branch in addition to the appearance of the apple. This makes it possible to improve the accuracy of estimating apple varieties.

Known varieties such as ``Shinano Gold'', ``Shuei'', and ``Shinano Pucci'' are used as the shooting target variety information. The imaging target product type information may indicate one type of product, or may indicate, for example, a plurality of product types and the probability of each product type.

In the sixth embodiment, it is possible to improve the convenience during the actual solution search by utilizing the imaging target variety information. When a user wearing an HMD implements the present invention after building the above-described learning data in advance, a search for search solutions (quality, sales price, etc.) is executed based on the method described in the above-described embodiment. . At this time, the imaged variety information may be obtained in the same way in the process of obtaining the appearance information of the imaged apple that the user wearing the HMD is visually recognizing. As a method of acquiring this photographing target variety information from the appearance information, it may be determined based on the learned feature amount as described above. It is also possible to estimate the photographing target variety information. In such a case, a machine learning model is used in which appearance information is used as teacher data, appearance information is input, and photographing target variety information is output. Then, based on the appearance information newly imaged via the user terminal, the imaging target variety information is acquired. As with the appearance information described above, it is also possible to acquire the photographing target variety information from the morphological information.

Hereinafter, the photographing target variety information acquired from the information including the appearance information, the morphological information, etc. is also referred to as the first photographing target variety information, and is obtained from the reference information including the reference appearance information, the reference morphology information, etc. The imaging target product type information is also referred to as second imaging target product type information.

Acquiring such first photographing target variety information each time in the process of acquiring appearance information and morphological information provides the excellent effects described below. It is possible to check each time whether or not the first photographed product type information in the photographed appearance information matches the second photographed product type information in the reference appearance information. When the first imaged product type information in the imaged appearance information and the second imaged product type information in the reference appearance information do not match, a user's attention can be displayed via the display unit of the HMD. In addition, when the first photographed product type information in the photographed appearance information and the second photographed product type information in the reference appearance information match, a display unit of the HMD is displayed to make the user recognize that they match. can be displayed.

For example, when the first photographed product type information in the photographed appearance information is “Shinano Gold” and the second photographed product information in the reference appearance information for matching is “Shuuei,” the HMD is The apple for which the user who wears it photographs the appearance information is photographing the wrong variety. In such a case, it is possible to urge the user to match the photographing object with the correct variety of apple by calling attention as described above.

In this way, the degree of matching between the first photographed product type information of the photographed appearance information and the second photographed product type information of the reference appearance information, or the photographed product type information of the photographed morphological information and the reference morphological information Via the degree of matching with the photographing target variety information, it is possible to make various suggestions to the user wearing the HMD about the method of actually photographing appearance information and morphological information, or prompt various corrections. . At this time, the above-described AR or MR may be realized in which such suggestions and promotion of correction are displayed in a transmissive state through the display unit of the HMD or glasses-type terminal. It should be noted that any suggestion about the photographing method may be displayed as long as it is based on the first photographing object product information and the second photographing object product information.

For example, as shown in FIG. 21, three or more degrees of association between reference appearance information or reference morphological information and apple quality may be acquired in advance for each apple variety (see FIG. 21). corresponding to “multiple valuation models”). In such a case, for each apple variety such as "for Shinano Gold", "for Shuei", "for Shinano Pucci", the degree of association between the appearance information for reference or the morphological information for reference and the quality of apples is independently calculated. Learn and save.

Then, the degrees of association of the apple varieties corresponding to the acquired first photographing object variety information are read out, and a solution search is performed in the same manner as described above. If the first photographed product type information is "Shinano Gold", an evaluation model consisting of the degree of association of "For Shinano Gold" is selected to estimate the quality. As a result, it is possible to select an evaluation model specialized for each apple variety, and perform more accurate quality estimation. In particular, by using an evaluation model that is specialized for each apple variety, it is possible to highly accurately determine whether or not the apple can be harvested when estimating the quality of the apple.

In the present invention, as shown in FIG. 22, the degree of association may be constructed for each apple variety as described above, and the degree of association may be classified for each reference cultivation history information. In such a case, for example, as shown in FIG. A degree of association is formed for each apple variety, such as "for". The degree of association for each apple variety, which is classified for each reference cultivation history information, is prepared. Then, when a new solution search is performed, the photographing object variety information described above is obtained, and the cultivation history information is obtained. Then, an evaluation model classified for each apple variety in the cultivation history corresponding to the cultivation history information is selected, and solution search is similarly performed.

Note that the degree of association constructed for each apple variety may be constructed for each piece of any other reference information, in addition to the case where it is constructed for each reference cultivation history information described above. In such a case, similarly, information corresponding to the reference information is obtained, and the degree of association is selected according to the obtained information.

According to the apple quality estimation model construction system in the present embodiment, the reference appearance information obtained by imaging the appearance of the apple after thinning, the information acquisition means for acquiring the quality of the apple, and the reference acquired by the information acquisition means A model generating means for generating an estimation model using appearance information and apple quality as training data, input as reference appearance information, and apple quality as output by machine learning. Further, the information acquiring means acquires shooting target variety information relating to the apple variety corresponding to the appearance of the apple, and records this in association with the reference appearance information. Therefore, it is possible to generate an estimation model for evaluating apple quality based on apple varieties. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation model construction system in the present embodiment, the information acquisition means for acquiring the reference morphological information obtained by imaging the appearance of the apple before fruit thinning and the quality of the apple, and the information acquisition means acquired by A model generating means for generating by machine learning an estimation model using the reference morphological information and the quality of the apple as teacher data, the input being the reference morphological information, and the output being the quality of the apple. Further, the information acquiring means acquires the shooting target variety information related to the apple variety, and records this in association with the reference morphological information. Therefore, it is possible to generate an estimation model for evaluating apple quality based on apple varieties. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

In addition, according to the apple quality estimation system of the present embodiment, it is included in the user terminal attached to the user's head or eyeglasses, and captures appearance information regarding the appearance of the apple to be estimated after thinning via the user terminal In addition, three or more levels of association are defined between information acquisition means for acquiring first imaging target variety information related to the imaged apple variety, reference appearance information imaged in the past for apples after thinning, and apple quality. using a trained model whose input is reference appearance information and whose output is apple quality, based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, and having a higher degree of association estimating means for estimating the quality of apples by prioritizing things. Further, the estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the apple for each of the above-mentioned apple varieties, and applies them to the first imaging target variety information acquired by the information acquiring means. The corresponding degree of association of apple varieties is read out to estimate the quality of the apple, and the user terminal has a display unit that transparently displays the quality of the apple estimated by the estimation means to the user. Therefore, it is possible to easily evaluate the quality of apples based on apple varieties. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation system of the present embodiment, the information acquisition means sets three or more degrees of association between the reference appearance information and the first imaging target variety information corresponding to the reference appearance information, Using a trained model whose input is reference appearance information and whose output is first photographing target product information, and based on the same or similar reference appearance information as appearance information, first photographing object product information corresponding to appearance information may be obtained. In this case, even a user who has difficulty discerning the quality of apples can acquire the apple variety (the first imaging target variety information) from the appearance information. As a result, highly accurate quality evaluation can be realized even when the user is not familiar with apple varieties.

Further, according to the apple quality estimation system of the present embodiment, the morphological information including the appearance of the apple to be estimated before thinning is imaged via the user terminal included in the user terminal worn on the user's head or glasses. In addition, at least three stages of information acquisition means for acquiring first imaging target variety information related to the imaged apple variety, reference morphological information imaged in the past including the appearance of the apple before fruit thinning, and the quality of the apple. Based on the same or similar reference morphological information as the morphological information acquired by the information acquisition means, using a trained model in which the degree of association is defined, the input is the reference morphological information, and the output is apple quality, an estimating means for estimating the quality of apples by prioritizing those with a higher degree of association. The estimating means preliminarily acquires three or more degrees of association between the reference morphological information and the apple quality for each apple variety, The quality of the apple is estimated by reading out the degree of association of the varieties of apples obtained, and the user terminal has a display unit that transparently displays the quality of the apple estimated by the estimation means to the user. Therefore, it is possible to easily evaluate the quality of apples based on apple varieties. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation system of the present embodiment, the information acquisition means sets three or more degrees of association between the reference morphological information and the first photographing target variety information corresponding to the reference morphological information, Using a trained model whose input is reference morphological information and whose output is first imaging target variety information, and based on the reference morphological information identical or similar to the morphological information, the first imaging target variety information corresponding to the morphological information may be obtained. In this case, even a user who has difficulty discerning the quality of apples can acquire the apple variety (the first imaging target variety information) from the appearance information. As a result, highly accurate quality evaluation can be realized even when the user is not familiar with apple varieties.

In addition, according to the quality estimation system of the present embodiment, the display unit displays information that is the same as or similar to the first imaging target variety information acquired by the information acquiring means and the appearance information (or morphological information) acquired by the information acquiring means. Based on the second photographing object product information corresponding to the reference appearance information (or the reference form information), the result of evaluating the degree of matching between the first photographing object product information and the second photographing object product information is Display in a transparent state. For this reason, it is possible to make various suggestions as appearance information (or morphological information) to the user who wears the user terminal, or prompt various corrections regarding the variety of the apple to be imaged. This makes it possible to proceed with the evaluation of apples more efficiently.

Seventh Embodiment Next, an example of an apple quality estimation model construction system and an apple quality estimation system in a seventh embodiment will be described. The difference between the above-described embodiment and the seventh embodiment is that an estimation model is generated and apple quality is evaluated based on cultivation history information. In addition, description is abbreviate|omitted about the content similar to embodiment mentioned above.

The cultivation history information indicates information about the cultivation history of apples, for example, the same information as in the above-described embodiment, for example, the cultivation history for the period before the apple tree bears fruit, and the entire cultivation up to harvesting. Includes history. Cultivation history information is acquired, for example, by a user's input so as to be associated with appearance information. Cultivation history information can be acquired, for example, by receiving it from a database storing cultivation histories, or it may be acquired by, for example, receiving information on the Internet. Cultivation history information may acquire position information, for example using GPS provided in the user terminal.

The cultivation history information includes, for example, branch pruning conditions. Branch pruning conditions can be specified, for example, from the appearance of apples before and after fruit thinning and the branches leading to them. For this reason, cultivation history information including branch pruning conditions can be obtained by using a well-known image analysis technique based on appearance information and morphological information obtained by photographing the appearance of apples and the appearance of branches leading to apples.

In addition, the cultivation history information may include, for example, deflowering conditions. Deflowering conditions can be specified, for example, from the shape of an apple. Therefore, cultivation history information including deflowering conditions can be obtained by using a well-known image analysis technique based on appearance information and morphology information obtained by photographing the shape of an apple.

For example, acquisition of cultivation history information may be determined based on previously learned feature amounts. For example, based on the appearance of an apple, deep learning techniques may be used to estimate quality through artificial intelligence. In such a case, a machine learning model that uses reference appearance information (or reference morphology information, the same applies hereinafter) and cultivation history information as teacher data, uses reference appearance information as input, and uses cultivation history information as output. use. Then, cultivation history information corresponding to the appearance information is acquired based on the appearance information (or morphological information, hereinafter the same) newly imaged via a user terminal or the like. The details of appearance information and morphological information and the acquisition method thereof are the same as in the above-described embodiment, and thus descriptions thereof are omitted.

In the seventh embodiment, by utilizing the cultivation history information, it is possible to enhance the convenience during the actual solution search. When a user wearing an HMD implements the present invention after building the above-described learning data in advance, a search for search solutions (quality, sales price, etc.) is executed based on the method described in the above-described embodiment. . At this time, cultivation history information may be obtained in the same way in the process of obtaining appearance information of an apple that is visually recognized by the user wearing the HMD. As a method of acquiring this cultivation history information from the appearance information, it may be determined based on the learned feature amount as described above. History information may be estimated. In such a case, a machine learning model is used in which appearance information is used as teacher data, appearance information is input, and cultivation history information is output. Then, the cultivation history information is acquired based on the appearance information newly imaged via the user terminal. It should be noted that cultivation history information can be obtained from morphological information in the same manner as the appearance information described above.

Hereinafter, cultivation history information acquired from information including appearance information, morphology information, etc., is also referred to as first cultivation history information. The information is also referred to as second cultivation history information.

Acquiring such first cultivation history information each time in the process of acquiring the appearance information and the morphology information has the excellent effects described below. It is possible to check each time whether the first cultivation history information of the imaged appearance information and the second cultivation history information of the reference appearance information match. When the first cultivation history information of the imaged appearance information and the second cultivation history information of the reference appearance information do not match, it is possible to display a warning to the user via the display unit of the HMD. Further, when the first cultivation history information in the imaged appearance information and the second cultivation history information in the reference appearance information match, a display for making the user recognize that they match is displayed via the display unit of the HMD. can do.

For example, when the first cultivation history information of the imaged appearance information is "cultivation history A" and the second cultivation history information of the reference appearance information for corresponding thereto is "cultivation history B", the HMD The user who wears the apple photographing the appearance information is photographing an incorrect cultivation history. In such a case, it is possible to urge the user to match the photographing object with an apple having a correct cultivation history by calling attention as described above.

In this way, the degree of matching between the first cultivation history information of the imaged appearance information and the second cultivation history information of the reference appearance information, or the cultivation history information of the imaged morphology information and the cultivation history of the reference morphology information Via the degree of matching with the information, it is possible to make various suggestions to the user wearing the HMD about the method of actually capturing the appearance information and the morphological information, or prompt various corrections. At this time, the above-described AR or MR may be realized in which such suggestions and promotion of correction are displayed in a transmissive state through the display unit of the HMD or glasses-type terminal. It should be noted that any suggestion about the photographing method may be displayed as long as it is based on the first cultivation history information and the second cultivation history information.

For example, as shown in FIG. 23, three or more degrees of association between reference appearance information or reference morphological information and apple quality may be acquired in advance for each apple cultivation history (FIG. 23). (corresponding to “Multiple Rating Models”). In this case, for each apple cultivation history such as “for cultivation history A”, “for cultivation history B”, and “for cultivation history C”, the reference appearance information or the reference form information and the apple The degree of association with quality is learned and stored.

Then, the degree of association of the apple cultivation history corresponding to the acquired first cultivation history information is read, and the solution search is performed in the same manner as described above. If the first cultivation history information is "cultivation history B", an evaluation model made up of the degree of association for "cultivation history B" is selected, and quality estimation is performed. As a result, it is possible to select an evaluation model specialized for each apple cultivation history, and to perform more accurate quality estimation. In particular, by using an evaluation model specialized for each cultivation history of apples, it is possible to highly accurately determine whether or not the apples can be harvested in accordance with the quality estimation of the apples.

According to the apple quality estimation model construction system in the present embodiment, the reference appearance information obtained by imaging the appearance of the apple after thinning, the information acquisition means for acquiring the quality of the apple, and the reference acquired by the information acquisition means A model generating means for generating an estimation model using appearance information and apple quality as training data, input as reference appearance information, and apple quality as output by machine learning. In addition, the information acquisition means acquires cultivation history information relating to the cultivation history of apples, and records this in association with the reference appearance information. Therefore, it is possible to generate an estimation model for evaluating the quality of apples based on the cultivation history of apples. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation model construction system in the present embodiment, the information acquisition means for acquiring the reference morphological information obtained by imaging the appearance of the apple before fruit thinning and the quality of the apple, and the information acquisition means acquired by A model generating means for generating by machine learning an estimation model using the reference morphological information and the quality of the apple as teacher data, the input being the reference morphological information, and the output being the quality of the apple. In addition, the information acquiring means acquires cultivation history information relating to the cultivation history of apples, and records this in association with the reference morphological information. Therefore, it is possible to generate an estimation model for evaluating the quality of apples based on the cultivation history of apples. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

In addition, according to the apple quality estimation system of the present embodiment, it is included in the user terminal attached to the user's head or eyeglasses, and captures appearance information regarding the appearance of the apple to be estimated after thinning via the user terminal In addition, three or more levels of association are defined between information acquisition means for acquiring first cultivation history information related to the cultivation history of the imaged apple, reference appearance information imaged of the apple after thinning in the past, and the quality of the apple. using a trained model whose input is reference appearance information and whose output is apple quality, based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, and having a higher degree of association estimating means for estimating the quality of apples by prioritizing things. Further, the estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the apple for each apple cultivation history, and uses the first cultivation history information acquired by the information acquiring means as The degree of association of the corresponding apple cultivation history is read to estimate the quality of the apple, and the user terminal has a display unit that transparently displays the quality of the apple estimated by the estimation means to the user. Therefore, it is possible to easily evaluate the quality of apples based on the cultivation history of apples. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation system of the present embodiment, the information acquisition means is set with three or more levels of association degrees between the reference appearance information and the first cultivation history information corresponding to the reference appearance information, and input is used as the reference appearance information and the output is the first cultivation history information, and the first cultivation history information corresponding to the appearance information is acquired based on the reference appearance information that is the same as or similar to the appearance information. may In this case, even a user who has difficulty discerning the quality of apples can acquire the apple cultivation history (first cultivation history information) from the appearance information. As a result, highly accurate quality evaluation can be realized even when the user is not familiar with the cultivation history of apples.

Further, according to the apple quality estimation system of the present embodiment, the morphological information including the appearance of the apple to be estimated before thinning is imaged via the user terminal included in the user terminal worn on the user's head or glasses. In addition, at least three levels of information acquisition means for acquiring first cultivation history information related to the cultivation history of the imaged apple, reference morphological information imaged in the past including the appearance of the apple before thinning, and the quality of the apple. Based on the same or similar reference morphological information as the morphological information acquired by the information acquisition means, using a trained model in which the degree of association is defined, the input is the reference morphological information, and the output is apple quality, an estimating means for estimating the quality of apples by prioritizing those with a higher degree of association. Further, the estimating means preliminarily acquires three or more degrees of association between the reference morphological information and the quality of the apple for each apple cultivation history, and according to the first cultivation history information acquired by the information acquisition means The quality of the apple is estimated by reading the degree of association of the cultivated history of the apples, and the user terminal has a display unit that transparently displays the quality of the apple estimated by the estimation means to the user. Therefore, it is possible to easily evaluate the quality of apples based on the cultivation history of apples. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation system of the present embodiment, the information acquisition means is set with three or more stages of association degrees between the reference morphological information and the first cultivation history information corresponding to the reference morphological information, and inputs is used as reference morphological information and the output is first cultivation history information, and the first cultivation history information corresponding to the morphological information is acquired based on the reference morphological information that is the same as or similar to the morphological information. may In this case, even a user who has difficulty in distinguishing the apple cultivation history can acquire the apple cultivation history (first cultivation history information) from the appearance information. As a result, highly accurate quality evaluation can be realized even when the user is not familiar with the cultivation history of apples.

Further, according to the quality estimation system of the present embodiment, the display unit displays the same or similar information as the first cultivation history information acquired by the information acquisition means and the appearance information (or morphological information) acquired by the information acquisition means. Based on the second cultivation history information corresponding to the appearance information for reference (or the morphological information for reference), the result of evaluating the degree of matching between the first cultivation history information and the second cultivation history information is displayed in a transparent state. do. For this reason, it is possible to make various suggestions or prompt various modifications to the user wearing the user terminal regarding the cultivation history of the apple taken as the imaging target as the appearance information (or morphological information). This makes it possible to proceed with the evaluation of apples more efficiently.

Eighth Embodiment Next, an example of an apple quality estimation model building system and an apple quality estimation system in the eighth embodiment will be described. The difference between the above-described embodiment and the seventh embodiment is that an estimation model is generated and the quality of apples is evaluated based on customer request information. In addition, description is abbreviate|omitted about the content similar to embodiment mentioned above.

Customer request information indicates information relating to requests from apple purchasers, and includes, for example, at least one of producer information, reviewer information, wholesaler information, and individual preference information.

The producer information indicates information about the producer of apples, and includes, for example, the name of the producer and the history of awards received at competitions. When producing apples, apple cultivation methods, harvest timings, etc. differ for each producer. Therefore, by evaluating apples based on producer information, it is possible to improve the accuracy when identifying apples of the quality demanded by apple purchasers.

For example, the producer information includes branch pruning conditions. Since branch pruning conditions differ from producer to producer, they can be identified, for example, from the appearance of apples before and after fruit thinning and the branches leading to them. For this reason, producer information including branch pruning conditions can be obtained by using well-known image analysis technology based on appearance information and morphological information obtained by photographing the appearance of apples and the appearance of branches leading to apples.

The producer information may also include, for example, flower picking conditions. Since the flower-thinning conditions are different for each producer, they can be specified, for example, from the shape of apples. For this reason, producer information including deflowering conditions can be obtained by using a well-known image analysis technique based on appearance information and morphological information obtained by photographing the shape of an apple.

The critic information indicates information about experts who criticize apples or dishes using apples, and includes, for example, the critic's name and review history. Since each reviewer has their own opinion, by evaluating apples based on the reviewer's information, it is possible to improve the accuracy when identifying apples of the quality required by apple purchasers. .

The wholesaler information indicates information about the wholesaler of the apples, and includes, for example, the name of the retailer such as a supermarket and the use of the purchased apples. Since the required quality of apples differs for each apple wholesaler, by evaluating apples based on the wholesaler information, it is possible to improve the accuracy when identifying apples with the quality required by apple purchasers. becomes possible.

The individual preference information indicates information about an individual who purchases apples, and includes, for example, the name of the individual and history of evaluations (eg, degree of satisfaction, number of stars, etc.) when apples were purchased in the past. Since the index of how apples are perceived as delicious varies greatly depending on individual preferences, by evaluating apples based on individual preference information, we will improve the accuracy of identifying apples of the quality required by apple purchasers. becomes possible.

The customer request information is acquired by, for example, inputting by the user so as to be associated with the appearance information. The customer request information can be obtained by receiving it from a database storing customer request information, for example, or by receiving information on the Internet, for example.

For example, wholesaler information or individual preference information may be acquired using personal authentication provided in the user terminal. In this case, the customer who purchases the apple can use the user terminal to directly specify the quality of the apple that the customer wants to purchase. As a result, even a customer who has no knowledge of apple evaluation can identify apples that meet the required quality without going through producers or wholesalers.

For example, acquisition of customer request information may be determined based on previously learned feature amounts. For example, based on the appearance of an apple, deep learning techniques may be used to estimate quality through artificial intelligence. In such a case, a machine learning model using external appearance information for reference (or morphological information for reference, the same shall apply hereinafter) and customer request information as training data, input as external appearance information for reference, and customer request information as output. use. Then, based on appearance information (or morphological information, the same shall apply hereinafter) newly captured via a user terminal or the like, customer request information corresponding to the appearance information is acquired. The details of appearance information and morphological information and the acquisition method thereof are the same as in the above-described embodiment, and thus descriptions thereof are omitted.

In the eighth embodiment, by utilizing the customer request information, it is possible to enhance the convenience during the actual solution search. When a user wearing an HMD implements the present invention after building the above-described learning data in advance, a search for search solutions (quality, sales price, etc.) is executed based on the method described in the above-described embodiment. . At this time, the customer request information may be obtained in the same way in the process of obtaining appearance information of an apple that is visually recognized by the user wearing the HMD. As a method of acquiring this customer request information from the appearance information, it may be determined based on the feature amount learned as described above. Request information may be estimated. In such a case, a machine learning model is used in which appearance information is used as training data, appearance information is input, and customer request information is output. Then, customer request information is acquired based on appearance information newly captured via the user terminal. It should be noted that customer request information can also be acquired from form information in the same manner as the appearance information described above.

Hereinafter, customer request information obtained from information including appearance information and form information is also referred to as first customer request information. The information is also referred to as second customer request information.

Acquisition of such first customer request information each time in the process of acquiring appearance information and form information provides the excellent effects described below. It is possible to confirm each time whether the first customer request information of the imaged appearance information and the second customer request information of the reference appearance information match. When the first customer request information of the imaged appearance information and the second customer request information of the reference appearance information do not match, a user's attention can be displayed via the display unit of the HMD. In addition, when the first customer request information of the imaged appearance information and the second customer request information of the reference appearance information match, a display for letting the user recognize that they match is displayed via the display unit of the HMD. can do.

For example, when the first customer request information of the imaged appearance information is "customer request A" and the second customer request information of the reference appearance information for matching is "customer request B", the HMD The user who wears the ringo shoots the external appearance information is shooting for the wrong customer's request. In such a case, it is possible to urge the user to match the photographing object with the apple corresponding to the correct customer request by calling attention as described above.

In this way, the degree of matching between the first customer request information of the photographed appearance information and the second customer request information of the reference appearance information, or the customer request information of the photographed form information and the customer request of the reference form information Via the degree of matching with the information, it is possible to make various suggestions to the user wearing the HMD about the method of actually capturing the appearance information and the morphological information, or prompt various corrections. At this time, the above-described AR or MR may be realized in which such suggestions and promotion of correction are displayed in a transmissive state through the display unit of the HMD or glasses-type terminal. It should be noted that any suggestion about the photographing method may be displayed as long as it is based on the first customer request information and the second customer request information.

For example, as shown in FIG. 24, three or more degrees of association between reference appearance information or reference shape information and apple quality may be acquired in advance for each apple customer request (FIG. 24). (corresponding to “Multiple Rating Models”). In such a case, for each customer request for apple such as "for customer request A", "for customer request B", "for customer request C", etc., the external appearance information for reference or the shape information for reference and the apple's The degree of association with quality is learned and stored.

Then, the degree of relevance of the customer request corresponding to the acquired first customer request information is read, and a solution search is performed in the same manner as described above. If the first customer request information is "customer request B", an evaluation model consisting of the degree of association for "customer request B" is selected to estimate quality. As a result, it is possible to select an evaluation model specialized for each customer request, and to perform more accurate quality estimation. In particular, by using an evaluation model that is specialized for each customer request, it is possible to accurately determine whether or not to harvest apples, which accompanies the quality estimation of apples.

According to the apple quality estimation model construction system in the present embodiment, the reference appearance information obtained by imaging the appearance of the apple after thinning, the information acquisition means for acquiring the quality of the apple, and the reference acquired by the information acquisition means A model generating means for generating an estimation model using appearance information and apple quality as training data, input as reference appearance information, and apple quality as output by machine learning. In addition, the information acquisition means acquires customer request information relating to the request of the purchaser of the apple, and records it in association with the appearance information for reference. Therefore, it is possible to generate an estimation model for evaluating the quality of apples based on customer requirements. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation model construction system in the present embodiment, the information acquisition means for acquiring the reference morphological information obtained by imaging the appearance of the apple before fruit thinning and the quality of the apple, and the information acquisition means acquired by A model generating means for generating by machine learning an estimation model using the reference morphological information and the quality of the apple as teacher data, the input being the reference morphological information, and the output being the quality of the apple. Further, the information acquiring means acquires customer request information relating to the request of the purchaser of the apple, and records it in association with the reference form information. Therefore, it is possible to generate an estimation model for evaluating the quality of apples based on customer requirements. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

In addition, according to the apple quality estimation system of the present embodiment, it is included in the user terminal attached to the user's head or eyeglasses, and captures appearance information regarding the appearance of the apple to be estimated after thinning via the user terminal In addition, information acquisition means for acquiring first customer request information related to the request of the purchaser of the imaged apple, the reference appearance information imaged of the apple after thinning in the past, and the degree of association with the quality of the apple in three or more stages is defined, using a trained model whose input is reference appearance information and output is apple quality, based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, the degree of association estimating means for estimating the quality of apples with higher priority. Further, the estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the apple for each customer request, The degree of relevance of the request is read to estimate the quality of the apple, and the user terminal has a display section for transparently displaying the quality of the apple estimated by the estimation means to the user. Therefore, it is possible to easily evaluate the quality of apples based on customer requirements. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation system of the present embodiment, the information acquisition means is set with three or more degrees of association between the reference appearance information and the first customer request information corresponding to the reference appearance information, and input is used as reference appearance information, and the output is the first customer request information, and the first customer request information corresponding to the appearance information is acquired based on the reference appearance information that is the same as or similar to the appearance information. may In this case, even a user who has difficulty discerning the quality of apples can acquire the customer's request (first customer's request information) from the appearance information. As a result, highly accurate quality evaluation can be realized even when the user is not familiar with the selection of apples for customer requirements.

Further, according to the apple quality estimation system of the present embodiment, the morphological information including the appearance of the apple to be estimated before thinning is imaged via the user terminal included in the user terminal worn on the user's head or glasses. In addition, information acquisition means for acquiring first customer request information related to customer requests of purchasers of photographed apples, reference morphological information photographed in the past including the appearance of apples before thinning, and quality of apples. Reference morphological information identical or similar to the morphological information acquired by the information acquisition means, using a trained model in which three or more levels of association are defined, the input being the reference morphological information, and the output being apple quality. estimating means for estimating the quality of apples by prioritizing those having a higher degree of association based on the above. Further, the estimating means preliminarily acquires, for each customer request, three or more degrees of association between the form information for reference and the quality of the apple, and obtains the customer's request according to the first customer request information acquired by the information acquiring means. The degree of relevance of the request is read to estimate the quality of the apple, and the user terminal has a display section for transparently displaying the quality of the apple estimated by the estimation means to the user. Therefore, it is possible to easily evaluate the quality of apples based on customer requirements. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

Further, according to the apple quality estimation system of the present embodiment, the information acquisition means is set with three or more degrees of association between the reference form information and the first customer request information corresponding to the reference form information, and inputs is used as reference form information, and the output is the first customer request information, and based on the same or similar reference form information as the form information, the first customer request information corresponding to the form information is acquired. may In this case, even a user who finds it difficult to distinguish apples that satisfy customer requirements can acquire the customer requirements (first customer requirement information) from the appearance information. As a result, highly accurate quality evaluation can be realized even when the user is not familiar with the selection of apples for customer requirements.

Further, according to the quality estimation system of the present embodiment, the display unit displays the same or similar information as the first customer request information acquired by the information acquisition means and the appearance information (or form information) acquired by the information acquisition means. Based on the second customer request information corresponding to the appearance information for reference (or the form information for reference), the result of evaluating the degree of matching between the first customer request information and the second customer request information is displayed in a transparent state. do. For this reason, it is possible to make various suggestions to the user wearing the user terminal regarding the customer's request for the apple taken as the imaging target as appearance information (or morphological information), or prompt various corrections. This makes it possible to proceed with the evaluation of apples more efficiently.

In addition, according to the quality estimation system in the present embodiment, it is included in the user terminal owned by the user, and images the appearance information related to the appearance of the apple to be estimated through the user terminal before or after the thinning, and the imaging Information acquisition means for acquiring the first customer request information related to the request of the purchaser of the apples that have been harvested, reference appearance information captured in the past for apples after thinning, and three or more levels of association with the quality of apples are defined. , using a trained model whose input is reference appearance information and whose output is apple quality, based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, and having a higher degree of association estimating means for estimating the quality of apples by prioritizing Further, the estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the apple for each customer request, Estimate apple quality by reading request relevance. The information acquisition means acquires first customer request information input from another user terminal. Therefore, the customer who purchases apples can input the first customer request information through another user terminal, and can select a favorite apple through the user terminal. Thereby, even if the customer is away from the apple, it is possible to easily evaluate the quality of the apple according to the customer's request. This makes it possible to automatically estimate the quality of apples with high accuracy without relying on human labor.

In the above-described embodiment, the case of estimating the quality of apples for fruits and vegetables has been described as an example, but the present invention is not limited to this. Any other fruits and vegetables can be similarly realized. In such a case, the description of "apples" in the above description is replaced with "fruits and vegetables", and the description below is omitted. In the case of other fruits and vegetables, solution search can be performed based on the same method as the apple quality estimation described above. In the case of the apple example, the pre-picking and post-picking processes have been described separately, but in the case of other fruits and vegetables, the pre-harvest and post-harvest conditions may also be subject to quality estimation.

1 apple quality estimation system 2 estimation device 21 internal bus 23 display unit 24 control unit 25 operation unit 26 communication unit 27 estimation unit 28 storage unit 61 node

Claims (5)

In a fruit and vegetable quality estimation system that estimates the quality of fruit and vegetables,
It is included in a user terminal worn on the user's head or eyeglasses, captures appearance information about the appearance of the fruit or vegetable to be estimated via the user terminal, and acquires first imaging target part information about the imaged part of the fruit or vegetable. an information acquisition means for
In the past, three or more levels of association were specified between the reference appearance information of the appearance of fruits and vegetables and the quality of the fruits and vegetables, and a trained model was used in which the input was the reference appearance information and the output was the quality of the fruits and vegetables. estimating means for estimating the quality of fruits and vegetables based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, giving priority to those having a higher degree of association;
recording means for pre-recording second imaging target part information related to parts of fruits and vegetables included in the reference appearance information in association with the reference appearance information;
The user terminal records the first imaging target part information acquired by the information acquisition means and the reference appearance information identical or similar to the appearance information acquired by the information acquisition means in the recording means in association with each other. A fruit and vegetable quality estimation system that notifies the user of a suggestion about an imaging method based on second imaging target part information that has been obtained. In a fruit and vegetable quality estimation system that estimates the quality of fruit and vegetables,
It is included in a user terminal worn on the user's head or eyeglasses, captures appearance information about the appearance of the fruit or vegetable to be estimated via the user terminal, and acquires first imaging target part information about the imaged part of the fruit or vegetable. an information acquisition means for
Using a learned model in which reference appearance information captured in the past about fruits and vegetables and three or more levels of association with the quality of the fruits and vegetables are defined, input is the reference appearance information, and output is the quality of the fruits and vegetables, estimating means for estimating the quality of fruits and vegetables based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, giving priority to those having a higher degree of association;
The estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the fruit or vegetable for each part of the fruit or vegetable, and obtains the first imaging target part acquired by the information acquiring means. estimating the quality of the fruits and vegetables by reading the degree of association of the parts of the fruits and vegetables according to the information;
The fruit and vegetable quality estimation system, wherein the user terminal notifies the user of the fruit and vegetable quality estimated by the estimation means. In a fruit and vegetable quality estimation system that estimates the quality of fruit and vegetables,
It is included in a user terminal worn on the user's head or eyeglasses, and captures appearance information regarding the appearance of a fruit or vegetable to be estimated via the user terminal, and acquires first imaging target variety information regarding the variety of the imaged fruit or vegetable. an information acquisition means for
Using a learned model in which reference appearance information captured in the past about fruits and vegetables and three or more levels of association with the quality of the fruits and vegetables are defined, input is the reference appearance information, and output is the quality of the fruits and vegetables, estimating means for estimating the quality of fruits and vegetables based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, giving priority to those with a higher degree of association,
The estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the fruits and vegetables for each variety of the fruits and vegetables, and obtains the first imaging target variety acquired by the information acquisition means. estimating the quality of fruits and vegetables by reading the degree of association of varieties of fruits and vegetables according to the information;
The fruit and vegetable quality estimation system, wherein the user terminal has a display section that displays the quality of the fruit and vegetable estimated by the estimation means to the user in a transparent state. In a fruit and vegetable quality estimation system that estimates the quality of fruit and vegetables,
It is included in a user terminal attached to a user's head or eyeglasses, and acquires first cultivation history information concerning the cultivation history of the imaged fruits and vegetables while capturing an image of appearance information regarding the appearance of the estimation target fruits and vegetables via the user terminal. an information acquisition means for
Using a learned model in which reference appearance information captured in the past about fruits and vegetables and three or more levels of association with the quality of the fruits and vegetables are defined, input is the reference appearance information, and output is the quality of the fruits and vegetables, estimating means for estimating the quality of fruits and vegetables based on reference appearance information identical or similar to the appearance information acquired by the information acquisition means, giving priority to those with a higher degree of association,
The estimating means preliminarily acquires three or more degrees of association between the reference appearance information and the quality of the fruit or vegetable for each cultivation history of the fruit or vegetable, and obtains the first cultivation history acquired by the information acquiring means. estimating the quality of the fruits and vegetables by reading the degree of association of the cultivation history of the fruits and vegetables according to the information;
The fruit and vegetable quality estimation system, wherein the user terminal has a display section that displays the quality of the fruit and vegetable estimated by the estimation means to the user in a transparent state. The fruit and vegetable quality estimation system according to any one of claims 1 to 5, wherein the degree of association is composed of nodes of a neural network in artificial intelligence.

Images