WO2022080504A1 - Determination device, learning device, determination method, learning method, and control program - Google Patents

Abstract

A determination device (1) according to an embodiment of the present invention comprises: an acquisition unit (12) that acquires an image including one or more grains of a granular agricultural product; and a determination unit (14) to which the image is input and that, using a learned machine learning model in which a determination result regarding the mycotoxin concentration or the degree of contamination in the grains of the granular agricultural product is output for each grain, determines whether individual grains or all of the one or more grains of the granular agricultural product are contaminated with a mycotoxin to a degree equal to or greater than a predetermined reference value or in a predetermined concentration range, or determines the mycotoxin concentration or the contamination degree of the individual grains or of all of the one or more grains of the granular agricultural product.

Description

Judgment device, learning device, judgment method, learning method and control program

The present invention relates to a determination device, a learning device, a determination method, a learning method, and a control program.

During the growing period or post-harvest storage period of cereals such as wheat and other crops, mycotoxins such as Fusarium head blight were infected or propagated, and part or all of the harvest was contaminated with mycotoxins. Things may get mixed in.

In order to prevent mycotoxins from contaminating cereals such as wheat due to Fusarium head blight, basically, by performing appropriate management (spraying chemicals at the appropriate time, etc.) at the cultivation stage in the field, the infection of Fusarium head blight itself can be prevented. It is important to prevent it as much as possible and prevent the accumulation of mycotoxins, but it may not always be possible to prevent it due to the influence of the weather conditions of the year.

Patent Document 1 discloses a color sorter that sorts red mold wheat by irradiating raw wheat with near-infrared light. Further, Patent Document 2 discloses a method of classifying the level of contamination in a cereal grain by performing multivariate data analysis on the diffused light absorption spectrum of the cereal grain.

Japanese Patent Publication "Japanese Patent Laid-Open No. 2005-28302" Japanese Patent Gazette "Special Table 2019-510968 Gazette"

Conventionally, especially in barley, the appearance symptoms of mycotoxins contaminated by Fusarium head blight are not clear, and although the effect of grain thickness selection is to some extent, the effect is not necessarily high, and the effect for reducing mycotoxins after harvesting. There was no high selection method. In wheat, mycotoxin-contaminated grains often show symptoms such as whitening and atrophy, and post-harvest grain thickness sorting, specific gravity sorting, and color sorting are effective in reducing mycotoxins to some extent, but in appearance. Mycotoxins may accumulate without overt symptoms, and there was no way to sort out such grains. In addition, when investigating the mycotoxin concentration or the degree of contamination of various crops such as harvested grains, it is necessary to crush the sample and use it for analysis by the commonly used chemical analysis method or immunological method such as ELISA, which is costly. -Since it takes time, a non-destructive and simple method for estimating the concentration of mycotoxins or the degree of contamination is required.

The inventor of the present application has decided to attempt image discrimination by machine learning using a data set of image data and mold poison concentration data for each grain of Fusarium head blight-infected barley grains whose appearance symptoms are generally not clear. As a result, contrary to expectations, it was shown that it is possible to distinguish between highly contaminated mycotoxins and non-contaminated grains at a level to be removed from the harvest. After that, it was shown that it is possible to distinguish between highly contaminated mycotoxins and uncontaminated or low-concentrated mycotoxins under other analysis conditions, and the findings of the present invention were obtained.

One aspect of the present invention is to improve the efficiency and accuracy in determination of mycotoxin concentration or degree of contamination of cereals such as wheat and other granular agricultural products, and selection and removal of mycotoxins contaminated grains based on the determination. do.

In order to solve the above problems, the determination device according to one aspect of the present invention includes an acquisition unit that acquires an image containing one or more grains of the granular agricultural product, and a mycotoxin in the grains of the granular agricultural product to which the image is input. Using a trained machine learning model that outputs the judgment result regarding the concentration or the degree of contamination for each grain, (1) each grain or the whole of one or more grains of the granular agricultural product is equal to or more than a predetermined reference value or a predetermined value. A determination unit for determining whether or not the product is contaminated with mycotoxins in the concentration range, or (2) determining the concentration or degree of mycotoxins in one or more individual grains or the entire grain of the granular agricultural product. Be prepared.

In order to solve the above problems, the learning device according to one aspect of the present invention includes an image containing one or more grains of the granular agricultural product, and concentration information indicating the mycotoxin concentration or the degree of contamination of each grain of the granular agricultural product. A machine learning model in which an image containing one or more grains of a granular agricultural product is input and a judgment result regarding the mycotoxin concentration or the degree of contamination in the grains of the granular agricultural product is output for each grain. It is provided with a learning unit for learning by using the set of the image and the density information acquired by the acquisition unit as teacher data.

In order to solve the above problems, the determination method according to one aspect of the present invention includes an acquisition step of acquiring an image containing one or more grains of the granular agricultural product, and a mycotoxin in the grains of the granular agricultural product to which the image is input. Using a trained machine learning model that outputs the judgment result regarding the concentration or the degree of contamination for each grain, (1) each grain or the whole of one or more grains of the granular agricultural product is equal to or more than a predetermined reference value or a predetermined value. It includes a determination step of determining whether or not the product is contaminated with mycotoxins in the concentration range, or (2) determining the concentration or degree of mycotoxins of one or more individual grains or the entire grain of the granular agricultural product.

In order to solve the above problems, the learning method according to one aspect of the present invention includes an image containing one or more grains of the granular agricultural product, and concentration information indicating the mycotoxin concentration or the degree of contamination of each grain of the granular agricultural product. The acquisition step is to acquire the machine learning model in which an image containing one or more grains of the granular agricultural product is input and the judgment result regarding the mycotoxin concentration or the degree of contamination in the grains of the granular agricultural product is output for each grain. It includes a learning step in which a set of the image and density information acquired in the step is used as teacher data for learning.

According to one aspect of the present invention, it is possible to improve the efficiency and accuracy in determining the mycotoxin concentration or the degree of contamination of cereals such as wheat and other granular agricultural products, and selecting and removing contaminated grains based on the determination.

It is a functional block diagram of the determination device (learning device) which concerns on embodiment of this invention. It is a figure which shows an example of the image of the grain unit of the harvested wheat. It is a flowchart which shows the flow of the determination processing example which concerns on embodiment of this invention. It is a flowchart which shows the flow of the learning processing example which concerns on embodiment of this invention. It is a figure which shows the information about the sample of wheat used in Example 1 and the like. It is a figure which shows the number of images for every contamination density | concentration of the wheat grain shown in FIG. It is a figure which shows the screen which displays the learning process of a neural network. It is a figure which shows the result of the determination process which concerns on Example 2. FIG. It is a figure which shows the result of the determination process which concerns on Example 3. FIG. It is a figure which shows the result of the determination process which concerns on Example 4. FIG. It is a figure which shows the result of the determination process which concerns on Example 4. FIG. It is a figure which shows the result of the determination process which concerns on Example 4. FIG. It is a figure which shows the information about the barley sample used as the teacher data and the test data set of Example 5. It is a figure which shows the learning process in the case of performing cross-validation using the image of barley of each sample series. It is a figure which shows the learning process in the case of performing cross-validation using the image of barley of each sample series. It is a figure which shows the result of the cross-validation which concerns on Example 5. It is a figure which shows the judgment result and the like when the image which becomes the test data set is given as the input to the trained neural network trained in cross-validation. It is a figure which shows the judgment result and the like when the image which becomes the test data set is given as the input to the trained neural network trained in cross-validation. It is a figure which shows the relationship between the probability threshold value which determines to be a highly contaminated grain and the selection yield for the test data set of the combination of each sample series in the cross-validation of Example 5. FIG.

[Embodiment]
Hereinafter, one embodiment of the present invention will be described in detail. In the present embodiment, the grain to be processed contained in the image is DON (deoxynivalenol), NIV (nivalenol), or the like, which is equal to or higher than a predetermined reference value or in a predetermined concentration range from a certain concentration to a certain concentration. A determination device capable of determining whether or not it is contaminated with mycotoxins will be described. Hereinafter, the subject will be described as being wheat as an example, but the subject is not limited to this, and as will be described later, the processing target of the determination device (learning device) may be other cereals such as corn. , Other various granular agricultural products may be used.

[1. Configuration example of judgment device (learning device)]
The determination device (learning device) 1 according to the present embodiment will be described. Further, in the following description, the names are not distinguished even when the determination device 1 functions as a learning device.

FIG. 1 is a functional block diagram of the determination device 1 according to the present embodiment. As shown in FIG. 1, the determination device 1 includes a control unit 10, a storage unit 20, an action unit 22, an imaging unit 24, a measurement unit 26, an input unit 28, and a display unit 30.

The control unit 10 is a control device that controls the entire determination device 1, and also functions as an acquisition unit 12, a determination unit 14, and a learning unit 16.

The acquisition unit 12 acquires an image containing one or more grains of the harvested wheat and concentration information indicating the mycotoxin concentration or the degree of contamination of each grain of the wheat.

In addition, the "mycotoxin concentration (mycotoxin concentration)" referred to here is the weight of the target mycotoxin contained in the target individual grain or the entire target grain (individual grain or multiple grains) in the entire target grain (individual grain or multiple grains). It is a value calculated as a ratio (target mycotoxin weight per unit weight), and the "mycotoxin degree" indicates the degree of mycotoxin contamination that can have a certain range, but includes the above-mentioned "mycotoxin concentration". In some cases.

FIG. 2 shows an example of the above image. The original image in FIG. 2 is an RGB color image, but is shown here in grayscale. The images included in the image group 50 of FIG. 2 show images of uncontaminated grains (barley) not contaminated with mycotoxins, and the images included in the image group 52 are images of mycotoxins contained in the grains (mycotoxins). Images of highly contaminated granules (barley) having a total of DON and NIV, which are trichothecene mycotoxins produced by Fusarium head blight, exceeding 5 ppm (μg / g) are shown. Further, in the image groups 50 and 52, the upper 6 images are the images on the front side of the grain, and the lower 6 images are the images on the back side of the grain. Here, the surface having the "grain groove" (also called "abdominal groove", which is a groove in the vertical direction) of the grain is referred to as the "back side", and the surface without the grain groove is referred to as the "front side". In addition, the grain of wheat such as barley and wheat has a grain groove (belly groove), and the front side and the back side are distinguished due to the structure of the grain, but the grain of corn and rice has no grain groove and is flat. When the grains are arranged on top, the distinction between the front side and the back side does not occur. In addition to barley and wheat, wheat includes barley (also referred to as oat and oat), rye, and triticale.

As a bulk sample of wheat (here, a set of grain samples is referred to as a bulk sample), an image containing tens to thousands of wheat grains, for example, one grain by the control unit 10. The configuration may be such that each image is divided and processed. Further, the images acquired by the acquisition unit 12 include an image for learning which is teacher data in the learning of the machine learning model, and an image for determination which is a determination target of the mold poison concentration or the degree of contamination of wheat grains in the image. There is an image. Further, in the present disclosure, the grains of wheat may be simply referred to as "wheat grains", and the grains of cereals and other granular agricultural products may be simply referred to as "cereals" or "grains". Further, the wheat grain may be barley or bare barley of two-row or six-row barley, wheat or the like. That is, the acquisition unit 12 can acquire, for example, an image containing one or more grains of harvested, peeled or peeled barley.

The determination unit 14 uses a trained machine learning model in which an image acquired by the acquisition unit 12 is input and outputs a determination result regarding the mycotoxin concentration or the degree of contamination in the wheat grains contained in the image, and the wheat grains are predetermined. Determine if it is contaminated with mycotoxins above the reference value or within the specified concentration range. Here, the determination result regarding the mycotoxin concentration or the degree of contamination in the wheat grain is, for example, when the wheat grain is classified according to the estimated value of the mycotoxin concentration in the wheat grain or the degree of contamination in each specific range, the wheat grain. May contain information indicating which class a is classified into.

The learning unit 16 uses a set of learning images acquired by the acquisition unit 12 and concentration information indicating the mold poison concentration or the degree of contamination of each grain of wheat as teacher data, and one or more grains of the harvested wheat grains. An image containing the above is input, and a machine learning model is trained in which the determination result regarding the mold poison concentration or the degree of contamination in the wheat grain is output for each grain. The determination method executed by the determination unit 14 and the machine learning method executed by the learning unit 16 are not limited to specific methods, and for example, CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), or the like is used. It may be. CNN and RNN are positioned as neural networks and deep learning (methods) among machine learning methods. Moreover, either classification or regression model may be used. The input data may be pre-processed and used for input to the machine learning model. For such processing, for example, when a neural network such as CNN is used, in addition to two-dimensional arrangement or multi-dimensional arrangement of data, various data augmentation (Data Augmentation), brightness, color tone, and Techniques such as adjustment of image quality and angle of an object, object detection for extracting an object area, background removal, and segmentation can be used.

When CNN is used, a convolution layer for performing a convolution operation is provided as one or more layers (layers) included in the neural network, and a filter operation (multiply-accumulate operation) is performed on the input data input to the layer. It may be configured to be performed. Further, when performing the filter calculation, a process such as padding may be used in combination, or an appropriately set stride width may be adopted.

In addition, for example, any of the following machine learning methods or a combination thereof may be used.

・ Support Vector Machine (SVM)
・ Decision tree
・ Random Forest
・ K-Nearest Neighbors (KNN)
・ Linear Discriminant Analysis (LDA)
・ Secondary discriminant analysis (QDA: Quadratic Discriminant Analysis)
・ Partial Least Squares Discriminant Analysis (PLSDA)
・ Simple Bayes (Naive Bayes)
・ Principal Component Analysis (PCA)
・ Regression Analysis
・ Ensemble Learning
・ Clustering
・ Inductive Logic Programming (ILP)
・ Genetic Programming (GP)
・ Bayesian Network (BN)
・ Multilayer perceptron (MLP)
・ Autoencoder
・ Transformer
The storage unit 20 is a storage device that stores various types of information, and is, for example, a parameter set that defines the machine learning model described above, and whether or not the wheat grains are contaminated with mycotoxins of a predetermined value or higher or a predetermined concentration range. Stores information and the like indicating a predetermined reference value as a reference. Further, for example, the storage unit 20 stores the captured image acquired by the acquisition unit 12 at least temporarily.

The acting unit 22 is a mechanism for aligning the target wheat grains in a predetermined direction based on the control by the control unit 10. For example, the acting unit 22 can be realized as an arm or a roller that exerts a physical action on the wheat grains that are stationary with respect to the ground plane.

The photographing unit 24 is a mechanism as a camera that photographs images such as wheat grains based on the control by the control unit 10. Further, the photographing unit 24 photographs an image of wheat grains in which the acting unit 22 aligns the orientation, and supplies the image to the acquisition unit 12. The photographing unit 24 may operate as an ultraviolet camera or an infrared camera, and may be configured to capture an image of wheat grains in the ultraviolet or infrared region. Further, the determination unit 14 may be configured to make a determination using a machine learning model learned by using an image containing one or a plurality of grains of wheat captured in the ultraviolet or infrared region as teacher data. .. As a result, the determination device 1 can perform more suitable determination processing by referring to information in the ultraviolet or infrared region that cannot be perceived by the naked eye. Further, by the action of the action unit 22 and the photographing unit 24, the acquisition unit 12 can acquire an image of the target wheat grain aligned in a predetermined direction.

The measuring unit 26 measures the mycotoxin concentration or the degree of contamination of the target wheat grain, estimates it by another method regardless of a chemical analysis method or an immunological method, and supplies the measurement result to the acquisition unit 12. The measurement result means concentration information indicating the mycotoxin concentration or the degree of contamination of wheat grains. As a method for estimating the degree of mycotoxin contamination of the target grain without using a chemical analysis method or an immunological method, for example, in wheat, when the target sample is contaminated with mycotoxin, the highly concentrated contaminated particles contained therein Since it may be possible to determine with the naked eye to some extent, it is assumed that the degree of mycotoxin contamination of the target grain is estimated by the naked eye or color selection to the extent possible. In addition, for example, the degree of mycotoxin contamination may be estimated by quantifying the amount of mycotoxin-producing bacteria in the target grain.

The input unit 28 is an interface for performing an operation or inputting information to the determination device 1. A part of the input unit 28 can be realized as a device such as a keyboard or a mouse that accepts an operation on the determination device 1.

The display unit 30 is a display panel that displays text, moving images, and the like based on the control of the control unit 10. The display unit 30 may be configured to realize some functions of the input unit 28 as a touch panel.

As another aspect, the processing target of the determination device 1 is not limited to wheat, but corn, rice (including paddy, brown rice, milled rice), miscellaneous grains (including buckwheat, dattan buckwheat, peanut, etc.), or Targets other cereals such as beans (including buckwheat) and granular agricultural products such as seeds and fruits of various crops including nuts or peanuts, coffee beans, cacao beans, nutmegs (seed or seeds), peppers, etc. The same treatment may be carried out, and the target granular agricultural product may be either with a shell or after removing the shell, and may be processed by grinding or the like. But it may be. For example, in the case of wheat, brown barley or refined barley may be used, and in the case of barley, malt, roasted grains for barley tea, round barley, pressed barley, rice grain barley (cut barley) or the like may be used. In addition, grains harvested before the optimum harvest time of each target crop may be targeted. In addition, various mold poisons other than DON / NIV due to Fusarium head blight and various mold poisons other than red mold (including aflatoxin, fumonisin, zearalenone, ochratoxin, T-2 toxin, HT-2 toxin, etc.), or mold poisons. Other than (including residual pesticides, arsenic, harmful heavy metals such as cadmium), diseases (including latent infections that are difficult to distinguish with the naked eye such as harmful microorganisms), and other poor quality traits are judged independently. Alternatively, the configuration may be such that a plurality of items are continuously or simultaneously performed. In the case of mycotoxins, individual grains or all of the multiple grains may be contaminated with multiple types of mycotoxins at the same time, but if necessary, these multiple mycotoxins may be added up for judgment. good. For example, DON and NIV, which are trichothecene-based (of which type B) mycotoxins, are particularly important as mycotoxins caused by Fusarium head blight in wheat, and these mycotoxins are chemically modified derivatives (various acetyls). The body and glycoxin) may accumulate in wheat grains at the same time as DON and NIV. If these substances are measured individually in association with DON or NIV in the grains used for teacher data, they may be treated together with individual DON or NIV as necessary (and at that time, of each substance). It may be added up using a converted value that reflects the ratio of the molecular weights of DON and NIV to the molecular weight). Further, the use of the granular agricultural product to be treated is not limited to food and feed, but may be for seeds, industrial use, and the like. In addition, after removing the target grains such as mycotoxin-contaminated grains to the extent possible by other methods such as color sorting in wheat where the appearance of the mycotoxin-contaminated grains can be distinguished to some extent, secondary sorting (judgment) is performed. ) The determination according to the present invention may be applied as a method. In addition, this judgment can be applied after sorting without images such as grain thickness sorting and specific gravity sorting, or after sorting inappropriate grains such as foreign substances, colored grains, and immature grains by other methods or image-based methods, or. At the same time, the determination according to the present invention may be applied. It can also be applied to the determination of various "good" traits in terms of quality, as opposed to the determination of bad traits. In addition, for example, target grains under conditions unsuitable for normal judgment, such as the water content exceeding the appropriate range, may be detected in advance or simultaneously by the judgment according to the present invention or another method, or the judgment may be corrected. You may.

In addition, the target is not necessarily limited to grains after harvesting, but for crops such as barley (barley) whose grains to be harvested are exposed at the time of fluffing in the field, for example, smartphones and drones in the field before harvesting. , An image taken by a camera mounted on a harvester or the like may be used to make a determination on the target grain contained in the image. In this case, image processing different from that at the time of determination of the object after harvesting, correction of determination, and the like may be performed.

Further, some functions of the photographing unit 24, the measuring unit 26, etc. in the determination device 1 may be realized on another device. In other words, the determination device 1 may be configured to be realized by a plurality of independent devices. For example, the device that performs the determination process and the device that performs the learning process may be realized as separate devices, and the devices may be interlocked with each other. Further, as another aspect, the determination device 1 may not be provided with the acting unit 22, and the user may manually perform the processing of the acting unit 22 as needed. The determination device 1 may be configured to be mounted on, for example, a sorting machine for sorting granular agricultural products, or may be configured to be realized as a device different from the sorting machine.

[2. Judgment processing example]
Subsequently, the determination device 1 refers to an image of grains of granular agricultural products to be determined (here, an example of wheat, which may be hereinafter referred to as wheat grains), and the wheat grains are equal to or higher than a predetermined reference value. An example of a determination process for determining whether or not mycotoxins are contaminated using a neural network (here, a regression model) as a machine learning model will be described. FIG. 3 is a flowchart showing the flow of processing in this example.

In step S101, the acting unit 22 performs a process of aligning the wheat grains to be determined by the determination device 1 in a predetermined direction. The predetermined orientation may be one or a plurality of orientations.

In step S102, the photographing unit 24 photographs an image of wheat grains in which the acting unit 22 aligns the directions. Further, the acquisition unit 12 acquires an image taken by the photographing unit 24.

In step S103, the determination unit 14 determines the orientation (front side, back side, etc.) of the wheat grains contained in the image acquired by the acquisition unit 12 with reference to, for example, the information supplied from the action unit 22, and then determines. After the unit 14 itself makes a determination with reference to the image, the image is input to the trained neural network according to the orientation of the wheat grain. Thereby, for example, a suitable neural network learned by using images of wheat grains having the same orientation can be used for the determination process. As mentioned above, unlike corn and rice, wheat grains such as barley and wheat have grain grooves (belly grooves) that can be clearly confirmed from the outside, and there is a distinction between the front side and the back side due to the structure of the grains. Therefore, it is considered that the grain in the image may be affected by whether it is the front side or the back side during learning or judgment. The "direction" of the wheat grain here may include not only the front side and the back side, but also the side surface, the tip side, the base side, and the directions having various angles and directions in which they can be confirmed on the image. I will do it.

In step S104, the determination unit 14 compares the determination result regarding the mycotoxin concentration of the wheat grain output from the neural network with the predetermined reference value stored in the storage unit 20, and the wheat grain is predetermined. Determine if it is contaminated with mycotoxins above the reference value or within the specified concentration range. Further, in this step S104, the control unit 10 causes the display unit 30 to display information indicating the determination result.

Note that the image input by the determination unit 14 to the trained neural network may contain a plurality of wheat grains. In this case, in the trained neural network, the determination regarding the mycotoxin concentration may be performed for each wheat grain, and the determination result may be output.

As described above, the determination method executed by the determination device 1 relates to the acquisition step of acquiring an image containing one or more grains of the harvested granular agricultural product, and the mycotoxin concentration in the grains of the granular agricultural product to which the image is input. Using a trained neural network that outputs the determination result for each grain, it is determined whether or not one or more grains of the granular agricultural product are contaminated with mycotoxins above a predetermined reference value or in a predetermined concentration range. An example of a determination method including a determination step has been described.

Further, as a subsequent process, for example, the determination unit 14 determines the degree of mycotoxin contamination and estimates the mycotoxin concentration by referring to one or more determination results by itself or the output of a plurality of neural networks. May be. Further, the comprehensive judgment may be performed using the judgment results of a plurality of orientations for each grain, or each based on a plurality of types of images (for example, a multispectral image in the visible light region and an image captured in the infrared region). Comprehensive judgment may be performed using the judgment results of different machine learning models. In addition, the determination unit 14 acquires an image containing tens to thousands of wheat grains as a bulk sample, and uses methods such as object detection, background removal, and segmentation to remove individual grains in the bulk sample. Is extracted or detected, and the mold poison concentration or the degree of contamination of each grain is determined. The degree of contamination may be determined or the mold poison concentration of the entire bulk sample may be estimated. Further, the determination unit 14 may determine whether or not one or a plurality of individual grains of the granular agricultural product or the entire bulk sample is contaminated with mycotoxins above a predetermined reference value or in a predetermined concentration range.

In step S103, the determination unit 14 determines whether the wheat seed (for example, wheat or barley, and if it is barley, whether it is barley or bare barley, or whether it is two-row barley or six-row barley. ) Or, after determining the direction of the barley grains, the image acquired from the acquisition unit 12 may be input to the trained neural network corresponding to each.

According to the configuration of this example, it is possible to improve the efficiency and accuracy of the determination regarding the mycotoxin concentration or the degree of contamination of wheat, for example, as compared with the determination with the naked eye.

<Modification example related to judgment processing>
The determination unit 14 has an estimated mycotoxin concentration of one or more individual grains of the granular agricultural product contained in the input image, which is equal to or higher than the predetermined reference value, or is equal to or higher than the predetermined reference value. If the probability of being contaminated with mycotoxins is equal to or higher than the set probability threshold, it is determined that the individual grains are contaminated with mycotoxins of the predetermined reference value or higher. May be good. The probability threshold here is a value of what percentage or more the probability that it is estimated to belong to a predetermined classification class when the target grain is determined by the trained machine learning model (in this case, the classification model). For example, the threshold value for classifying into the relevant class is shown. More specifically, the probability threshold here is, for example, if the probability that the target barley grain is estimated (classified) as a highly contaminated grain having a predetermined reference value or more is 0% or more, the barley grain is concerned. Indicates the threshold value for determining whether is a highly contaminated grain.

In a broad sense, the determination unit 14 determines whether or not one or a plurality of individual grains of the granular agricultural product are contaminated with mycotoxins above a predetermined reference value using a predetermined determination threshold value. Here, the predetermined reference value and the probability threshold value are examples of the determination threshold value. The determination threshold value may be set by the determination unit 14.

Further, in the machine learning model such as a neural network in the above configuration, an image containing one or more grains of the granular agricultural product is input, and the individual grains of the one or more grains of the granular agricultural product have a predetermined reference value or more. It may be configured to output the probability of being contaminated with mycotoxins.

Further, the determination unit 14 may calculate the selection yield corresponding to each of one or more determination threshold values for the set of grains to be determined. Here, the sorting yield is determined by the determination unit 14 that the set of grains to be determined, that is, one or more grains of the granular agricultural product, is not contaminated with mycotoxins of a predetermined reference value or more. The ratio of grains (ratio of number of grains or ratio of number of grains images) or its weight ratio.

For example, the sorting yield (grain number ratio) (%) is calculated by the formula of (1- (the number of grains determined by the determination device to be highly contaminated grains / the number of grains to be determined)) × 100. Further, for example, as the teacher data, the grain weight data corresponding to each grain image, or the average grain weight of low-concentration contaminated grains or uncontaminated grains less than a predetermined reference value in the same sample lot of the grain image is used as a reference. By learning the data of the relative grain weight ratio (of individual grains) at that time as a set with each grain image, each grain weight or low concentration (less than a predetermined reference value) is obtained from each grain image to be determined. It is also possible to estimate the relative grain weight ratio (to contaminated or uncontaminated grains). From this, using the estimated value, the sorting yield (weight ratio) (%) is set to (1- (total grain weight (estimated value) of the grain image determined by the judgment device as a highly contaminated grain / judgment target). (Total grain weight (estimated value) of whole grain image)) × 100 and (1- (Total value of relative grain weight ratio (estimated value) of grain image judged as high-concentration contaminated grain by the judgment device / judgment target It may be calculated (estimated) by a formula of (total value)) × 100 of the relative grain weight ratio (estimated value) of the whole grain image. Here, it is assumed that the relative grain weight ratio data tends to be smaller than 1 in the mycotoxin-concentrated grains, for example, in wheat, which is generally considered to have a large effect of reducing mycotoxins by specific weight sorting. ..

Alternatively, for example, the formula of the sorting yield (grain number ratio) (%) is corrected by using the coefficient A assumed or estimated from the data accumulated so far, and the sorting yield (weight ratio) (%) is obtained. , (1- (Number of grains determined by the determination device to be highly contaminated grains × A / Number of grains to be determined)) × 100 can also be estimated. Further, by providing the determination device 1 with a mechanism for directly measuring the weight of each grain before or after acquiring the image of each grain to be determined, the sorting yield is used by using the weight data of each grain directly measured. (Weight ratio) (%) is more accurate with the formula of (1- (total actual weight of grains judged by the judgment device to be highly contaminated grains / total actual weight of grains to be judged)) × 100. It can also be calculated as a value. Such a value of the sorting yield (grain number ratio or weight ratio) can be used as a judgment material when determining a judgment threshold value to be used for determining high-concentration contaminated grains, for example. It should be noted that the sorting yield based on the grain number ratio is easier to calculate than the sorting yield based on the weight ratio, but in general, it directly indicates how much the yield remains as the weight ratio after grain sorting. It is considered that the sorting yield based on the weight ratio is often more important when examining the judgment threshold value in the actual grain sorting scene.

Further, as will be described later, the determination unit 14 is based on an estimation model developed using teacher data or the like accumulated for improving the determination performance of the determination device, from a set of grains to be determined for the granular agricultural product. , Mycotoxin reduction rate or the degree of mycotoxin concentration or degree of contamination of the entire set of remaining grains when grains judged to be contaminated with mycotoxins above a predetermined standard value are removed using a predetermined judgment threshold value. May be estimated for each of one or more of the determination thresholds or selection yields.

[3. Learning process example]
Next, an example of a learning process in which the determination device 1 learns a neural network as a machine learning model using teacher data will be described by taking the case of wheat as an example. FIG. 4 is a flowchart showing the flow of processing in this example.

In step S201, the acting unit 22 performs a process of aligning the wheat grains in a predetermined direction. The predetermined orientation may be one or a plurality of orientations.

In step S202, the photographing unit 24 photographs an image of wheat grains in which the acting unit 22 aligns the directions. Further, the acquisition unit 12 acquires an image taken by the photographing unit 24. After that, the process of returning to step S201, aligning the same wheat grains in different directions, and then taking an image again in this step may be performed as many times as necessary.

In step S203, the measuring unit 26 measures or estimates the mycotoxin concentration or the degree of contamination of the wheat grains used in the learning process. Further, the acquisition unit 12 acquires the value of the mycotoxin concentration or the degree of contamination as the concentration information.

In step S204, the learning unit 16 determines the orientation (front side, back side, etc.) of the wheat grains contained in the image acquired by the acquisition unit 12 by the user via, for example, information supplied from the action unit 22 or the input unit 28. After making a determination with reference to the input information indicating the orientation of the wheat grains, the set of the image and the density information of the wheat grains acquired by the acquisition unit 12 is used as teacher data in the neural network corresponding to the orientation of the wheat grains. To train the neural network. In the above neural network, an image containing one or more grains of harvested wheat is input, and a determination result regarding the mycotoxin concentration or the degree of contamination in the grains of the wheat is output for each grain.

The image input by the learning unit 16 to the neural network may contain a plurality of wheat grains. In this case, in addition to the wheat grain image and the density information of each wheat grain, information such as coordinates indicating the position of each wheat grain in the image may be input to the neural network.

As described above, it is a learning method executed by the learning device (judgment device) 1, and is an image containing one or more grains of the harvested granular agricultural product and a concentration indicating the mold poison concentration or the degree of contamination of each grain of the granular agricultural product. A neural network in which an acquisition step for acquiring information and an image containing one or more grains of harvested granular agricultural products are input, and judgment results regarding the fungal poison concentration or the degree of contamination in the grains of the granular agricultural products are output for each grain. An example of a learning method including a learning step in which a set of the image and density information acquired in the acquisition step is used as teacher data for learning has been described.

In step S204, the learning unit 16 determines the direction of wheat seeds (for example, wheat or barley, and in the case of barley, whether it is barley or bare barley, two-row barley or six-row barley) and wheat grains (for example, wheat or barley. The front side, the back side, etc.) are discriminated by referring to the information indicating the barley type, etc. input by the user, for example, via the input unit 28, and then the image acquired from the acquisition unit 12 is applied to the neural network corresponding to each. May be configured to input.

Further, in step S204, the learning unit 16 may perform preprocessing such as clipping or resizing the image acquired by the acquisition unit 12. Further, the learning unit 16 performs data segmentation such as inversion, enlargement, reduction, and translation of the image acquired from the acquisition unit 12, and adjusts the brightness, color tone, image quality, angle of the object, and the like. , The target area may be extracted by object detection, background removal, segmentation, or the like. Further, the extraction of the target region is not necessarily limited to the entire grain, and only a part of the grain that should be noted may be extracted.

According to the configuration of this example, it is possible to learn a neural network for improving the efficiency and accuracy in determining the mycotoxin concentration or the degree of contamination of wheat.

Note that the learning unit 16 may use only images of grains corresponding to one or a plurality of specific classes as teacher data when grains of wheat or the like are classified according to the degree of contamination for each specific range. For example, the learning unit 16 includes an image containing grains of the first class in which the mycotoxin concentration or the degree of contamination is equal to or higher than a predetermined reference value, and the mycotoxin concentration or the degree of contamination is equal to or less than a predetermined threshold set to be less than the reference value. (For example, 70% or less of the standard value, half or less of the standard value, or one-fifth or one-tenth or less of the standard value, etc.) or non-contaminated grains of second-class granular agricultural products. The containing image is input to the machine learning model, and the machine learning model is used, for example, as a result of determining whether or not the grains contained in the input image are grains contaminated with mycotoxins having a predetermined reference value or more. It may be configured to be trained to be output as a probability value.

Further, the above-mentioned predetermined reference value and predetermined threshold value are not limited to specific values and may be changed. For example, a predetermined reference value may be 5 ppm and a predetermined threshold value may be 0.5 ppm.

At this time, by using grains contaminated with mold poison above the reference value and grains having a significantly different degree of contamination for learning, for example, the values of the parameter set that defines the machine learning model are suitably converged, and the machine learning model is used. The accuracy of the output value of can be improved. In addition, by using not only non-contaminated particles but also low-concentration contaminated particles below a predetermined threshold for learning, the present invention can be applied to, for example, an actual mold poison-contaminated sample (grains having various degrees of contamination from uncontaminated to high-concentration contamination). When applied to the removal of high-concentration contaminated particles from (possible), acceptable low-concentration contaminated particles can be distinguished from high-concentration contaminated particles and left in the sample. It is considered that the weight ratio to the sample or the number of grains ratio) does not need to be lowered more than necessary.

As teacher data used when learning such two-class classification, the threshold of the fungal poison concentration when classifying the high-concentration side class and the low-concentration contaminated to non-contaminated side class is set to the high concentration side (predetermined here). (Reference value) and the low concentration side, respectively, and when using the machine learning model after learning for discrimination (grain selection), adjust the probability threshold when determining whether or not it is a high concentration contaminated grain. It is considered that the balance between the mold poison reduction effect by removing high-concentration contaminated grains and the sorting yield (these are usually in a trade-off relationship) can be adjusted. As an example of the configuration of the determination device 1, the determination unit 14 may determine the mycotoxin concentration or the degree of contamination using the above probability threshold value that can be adjusted by input to the input unit 28 by the user or the like.

Further, for example, in wheat in which the appearance of highly contaminated particles of mycotoxins can be distinguished to some extent, the primary sorting is performed when the determination according to the present invention is applied as the secondary sorting method after performing the primary sorting by color sorting or the like. It is also possible to perform more efficient selection by using a machine learning model trained with a threshold setting different from that in the case where it is not performed. Further, it is also possible to apply the technique of the present invention by setting a threshold value according to each stage to all of the selection of a plurality of stages such as the primary selection and the secondary selection, and perform efficient selection. It is also possible to perform sorting by such a multi-step machine learning model continuously at the time of one sorting.

[Example of implementation by software]
The control block (particularly, the acquisition unit 12, the determination unit 14, and the learning unit 16) of the determination device (learning device) 1 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. , May be realized by software.

In the latter case, the determination device 1 includes a computer that executes a program instruction, which is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes the program, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a “non-temporary tangible medium”, for example, a ROM (Read Only Memory) or the like, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (RandomAccessMemory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. It should be noted that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

[Example 1]
An embodiment of the present disclosure will be described below. FIG. 5 shows information about the wheat samples used in this example and subsequent examples. In FIG. 5, the “target grain thickness fraction” indicates the range of the thickness of each target grain. Further, the "sample series" indicates the lot number in the corresponding "variety", and is a sample obtained under different conditions. All varieties are domestically cultivated varieties, and the distinction between varieties is indicated by alphabets (3 varieties of wheat and 2 varieties of barley).

The "Fusarium head blight infection condition" is either a "natural infection" in which wheat grains are naturally infected with Fusarium head blight, or a source of infection is sprayed on the field to artificially "promote" the infection of Fusarium head blight. It shows whether it is an infection. Further, the "mycotoxin concentration of the fraction" indicates the mycotoxin concentration per unit amount in the entire sample fraction of the sampling source of the wheat grain sample of the corresponding "sample series". The items shown on the right side of the "mycotoxin concentration in the fraction" indicate the number of wheat grains at each contamination concentration. Here, the mycotoxin concentration is the total value of DON and NIV measured by ELISA.

Further, FIG. 6 is an image used as a data set in this example and subsequent examples, and shows the number of images for each contamination concentration of wheat grains shown in FIG. As shown in FIGS. 5 and 6, for each grain of FIG. 5, 1 to 3 images correspond to each of the front side and the back side of the wheat grain. All the images are images taken by a general digital camera.

FIG. 6 shows, for example, that a total of 149 images of the front side of each contamination concentration and 148 images of the back side of each contamination concentration were prepared for the image of wheat of variety A, and the image of barley exceeding 5 ppm was prepared on the front side of each sample series. It shows that a total of 39 images of the above and 39 images of the back side were prepared. In addition, a part of the image of wheat grains of each contamination concentration is separated from the image used for learning as a test data set which is a test image (an image derived from different wheat grains for learning and testing). In addition, during learning, the data is amplified by flipping left and right in the class with a small number of images so that the number of images is almost the same between the classification classes or within about 1: 2, and the other class is appropriately imaged. The number was reduced and used.

In the determination device according to the determination device 1 described above in the above embodiment, each image data classified into high-concentration contaminated grains of barley exceeding 5 ppm and N.D., that is, uncontaminated grains of about 0 ppm, and the degree of contamination of each barley are taught. As data, we trained a neural network using AlexNet. In this example and the examples described later, Nijo barley was used as the barley, but the same treatment can be applied to other barley and other granular agricultural products such as grains. Is. At the time of learning, the teacher data was divided into learning data and verification data at 8: 2, and the image for learning was subjected to position movement and left-right inversion data augmentation.

FIG. 7 shows a screen displaying the learning process. In FIG. 7, the “loss” corresponds to the value of the loss function in learning, and converges to a value close to 0 by repeating the learning. Further, the "learning rate" in FIG. 7 means how much each value of the parameter set defining the neural network is adjusted in one learning.

As shown in FIG. 7, as a result of repeating the learning of the data divided into mini-batch for each epoch 16 times for barley, learning up to the number of epochs and testing, the verification accuracy by the verification data at the time of learning was 92%, and the test data. The test accuracy of the set (highly contaminated grain image and non-contaminated grain image separated from the learning image in advance) is 87% (27/31). ), And the ratio (correct answer rate) that could be estimated with high accuracy as a high-concentration contaminated grain with a probability of 90% or more in the test image of the high-concentration contaminated grain exceeding 5 ppm was 61% (11/18).

Similarly, as a result of separately learning up to the number of epochs 40, the verification accuracy is 97%, the test accuracy is 84% (26/31), and the correct answer rate in the above judgment result of the highly concentrated contaminated particles is 67% (12/18). It became.

In addition, in the determination device, each image data classified into high-concentration contaminated grains of barley exceeding 5 ppm and barley (low-concentration contaminated grains) of 0.05 to 0.5 ppm, which is an allowable level, and contamination of each barley. Using the degree as teacher data, we trained a neural network using AlexNet in the same way. At that time, the learning of the data divided into mini-batch for each epoch was repeated 16 times, and the learning was performed up to the number of epochs of 40. As a result, the verification accuracy is 90%, and it is either a high-concentration contaminated grain or a low-concentration contaminated grain in the test data set (high-concentration contaminated grain image and low-concentration contaminated grain image separately separated from the learning image in advance). The ratio of high-concentration contaminated particles with a high probability of being highly contaminated with a probability of 90% or more among the test images of highly contaminated particles with an alternative test accuracy of 86% (38/44) or more than 5 ppm (correct answer rate). Was 72% (13/18).

As mentioned above, even barley, which is more difficult to distinguish mycotoxin-contaminated grains by appearance, can be classified as high-concentration contaminated grains and non-contaminated grains or (acceptable level) low-concentration grains with an accuracy of 80% or more. It was confirmed that it is possible to distinguish between.

[Example 2]
A second embodiment of the present disclosure will be described below. In this example, among the image data shown in FIG. 6, each image data classified into high-concentration contaminated grains of barley exceeding 5 ppm and non-to low-concentrated contaminated grains of barley less than 0.5 ppm, and barley. Using the degree of contamination of each grain as training data, a neural network using AlexNet was trained a sufficient number of times.

In addition, as a test data set (including images of contaminated grains outside the extreme two classes of fungal poison concentration range used for learning, assuming an actual mold poison contamination sample), barley exceeding 5 ppm (high concentration contaminated grains). 18 images, 17 images of 1-5 ppm barley (slightly contaminated grains), 12 images of 0.05-1 ppm barley, and 14 images of uncontaminated barley. 61 sheets were used.

Explanation of FIG. 8 FIG. 56 shows the test result of determining whether or not the above test data set is “high-concentration contaminated particles”. Further, in the explanatory diagram 56 and each of the subsequent figures, "high" corresponds to high-concentration contaminated grains exceeding 5 ppm, and "Not" corresponds to other wheat grains. Further, "true" means whether or not the particles are actually highly contaminated particles, and "estimation" means the judgment result by the judgment device. In the judgment, when the target barley grain is estimated to be a highly contaminated grain having a probability of exceeding 5 ppm with a probability of 50% or more, it is determined that the barley grain is a highly contaminated grain. bottom. In other words, the probability threshold for determining high-concentration contaminated particles was set to 50%.

As shown in the explanatory diagram 56, among the images of the barley grains determined by the determination device to be highly contaminated grains, the correct answer is 8 images, the false detection is 2 images, and the images of the barley grains determined to be other than that. Of these, 10 images were misjudgments that were overlooked, and 41 images were correct answers. As a result, the test accuracy (ratio of the number of correct answers) became 80%. This result shows that out of 18 highly contaminated grain images, when an attempt was made to sort and remove (images of) highly contaminated grains with this determination device for a test data set containing images of grains of various degrees of contamination. Eight images could be sorted and removed, and images of grains with a degree of contamination of less than 5 ppm mean that 41 of 43 could be left unremoved. There were two images of false detection, but these were images of contaminated particles with a slightly high concentration (1 to 5 ppm), and these were also particles that could be removed.

Further, the explanatory diagram 58 shows the determination result when the same learning and determination are performed using Xception as a neural network. In the above case, among the images of the barleycorn determined by the determination device to be highly contaminated grains, the correct answer is 8 images, the false detection is 3 images, and among the images of the barleycorn determined to be other than that, among the images of the barleycorn. The wrong judgment was 10 images, and the correct answer was 40 images. As a result, the test accuracy was 79%. In addition, 2 out of 3 images of false detection in this result were images of contaminated particles having a slightly high concentration (1 to 5 ppm).

Further, the explanatory diagram 60 shows the determination result when the same learning and determination are performed using VGG16 as a neural network. In the above case, among the images of the barley grains determined by the determination device to be highly contaminated grains, the correct answer is 8 images, the false detection is 9 images, and among the images of the barley grains determined to be other than that, among the images of the barley grains. The wrong judgment was 10 images, and the correct answer was 34 images. As a result, the test accuracy was 69%.

Further, the explanatory diagram 62 shows the determination result when the same learning and determination are performed using ResNet50 as a neural network. In the above case, among the images of barley grains determined by the determination device to be high-concentration contaminated grains, the correct answer is 16 images, the false detection is 16 images, and the determination device determines that the barleycorn is non-low-concentration contaminated grains. Of the images of the barleycorns that were made, 2 images were misjudged and 27 images were correct. As a result, the test accuracy was 70%.

As described above, in this embodiment corresponding to FIG. 8, when any of AlexNet, Xception, VGG16, and ResNet50 is used, image classification of high-concentration contaminated grains exceeding 5 ppm and non-low-concentration contaminated grains of less than 0.5 ppm of barley is used. The learning results in were converged, and the test accuracy in determining high-concentration contaminated grains in the test data set containing images of grains with various mold poison contamination concentrations was 69-80%, but when AlexNet or Xception was used. A relatively high test accuracy was obtained.

[Example 3]
A third embodiment of the present disclosure will be described below. In this example, among the image data shown in FIG. 6, about 176 images each classified into high-concentration contaminated grains of wheat exceeding 5 ppm and non-to low-concentrated contaminated grains of wheat of less than 0.5 ppm. Using the data and the degree of contamination of each grain of wheat as training data, a neural network using AlexNet was trained a sufficient number of times.

In addition, as a test data set (including images of contaminated grains outside the extreme two classes of fungal poison concentration range used for learning, assuming an actual mold poison contamination sample), wheat exceeding 5 ppm (high concentration contaminated grains). 23 images, 29 images of 1-5 ppm wheat (slightly contaminated grains), 24 images of 0.05-1 ppm wheat, and 26 images of uncontaminated wheat. 102 sheets were used.

Explanation of FIG. 9 FIG. 64 shows the test results of determining whether or not the test data set is “high-concentration contaminated particles”. However, in the determination shown in the explanatory diagram 64, the probability threshold value for determining the high-concentration contaminated grains was set to 50% as in Example 2 for barley. As a result, among the images of wheat grains judged to be highly contaminated grains by the judgment device, the correct answer is 19 images, the false detection is 21 images, and among the images of the wheat grains judged to be other than that, the wrong judgment is made. There were 4 images and 58 correct answers. As a result, the test accuracy was 75%. Of the 21 images of the above false detection, 16 images were contaminated particles having a slightly high concentration of 1 to 5 ppm.

Further, the explanatory diagram 66 shows the determination result when the same learning and determination are performed using Xception as a neural network. Further, in the determination shown in the explanatory diagram 66, the probability threshold value for determining the high-concentration contaminated particles was set to 50%. As a result, among the images of wheat grains judged to be highly contaminated grains by the judgment device, the correct answer is 21 images, the false detection is 25 images, and among the images of wheat grains judged to be other than that, the wrong judgment is made. There were 2 images and 54 correct answers. As a result, the test accuracy was 74%. Of the 25 images of the above false detection, 17 images were contaminated particles having a slightly high concentration of 1 to 5 ppm.

Further, FIG. 68 shows a determination result when the probability threshold value for determining a high-concentration contaminated grain is set to 97% instead of 50% when learning and determination are performed using AlexNet as a neural network. ing. In this case, among the images of wheat grains determined by the determination device to be highly contaminated grains, the correct answer is 16 images, the false detection is 8 images, and among the images of wheat grains determined to be other than that, the wrong judgment is made. Was 7 images and the correct answer was 71 images. As a result, the test accuracy was 85%. In addition, all of the eight images of the above false detection were contaminated particles having a slightly high concentration of 1 to 5 ppm.

Further, FIG. 70 shows a judgment result when the probability threshold value for judging as a highly concentrated contaminated grain is set to 85% instead of 50% when learning and judgment are performed using Xception as a neural network. ing. In this case, among the images of wheat grains determined by the determination device to be highly contaminated grains, the correct answer is 15 images, the false detection is 5 images, and among the images of wheat grains determined to be other than that, the wrong judgment is made. Was 8 images and the correct answer was 74 images. As a result, the test accuracy was 87%. Of the 5 images of the above false detection, 4 images were contaminated particles having a slightly high concentration of 1 to 5 ppm.

That is, in this example (wheat), the accuracy of the determination could be improved by adjusting the probability threshold value for determining the high-concentration contaminated grains.

[Example 4]
A fourth embodiment of the present disclosure will be described below. In this embodiment, a configuration using a neural network according to the orientation of the barley grains in the image input to the determination device will be described. In this example, the same teacher data and test data set as in Example 2 were used as described later. In addition, the probability threshold for determining high-concentration contaminated particles was set to 50%.

Explanation of FIG. 10 FIG. 72 shows test data of the front side image of barley when the neural network using AlexNet is trained a sufficient number of times by mixing the front side image and the back side image of barley which are teacher data. The judgment result when used as a set is shown. Explanatory The test accuracy of the determination shown in FIG. 72 was 82%. In addition, one image of erroneous detection in the explanatory diagram 72 was a contaminated particle having a slightly high concentration of 1 to 5 ppm. Further, FIG. 76 shows the determination result when the image of the back side of the barley is used as a test data set in the above case. Explanatory The test accuracy of the determination shown in FIG. 76 was 79%.

Further, FIG. 74 shows the determination result when the front side image of barley is used as a test data set when the neural network using AlexNet is trained a sufficient number of times using only the front side image of barley as teacher data. Shows. Explanatory The test accuracy of the determination shown in FIG. 74 was 82%.

Further, FIG. 78 shows the determination result when the image of the back side of the barley is used as the test data set when the neural network using AlexNet is trained a sufficient number of times using only the image of the back side of the barley as the teacher data. Shows. Explanatory The test accuracy of the determination shown in FIG. 78 was 65%.

As described above, in the results of learning with AlexNet in FIG. 10, when the front side image and the back side image of barley are mixed in the teacher data, and when only the front side and back side images are used for the teacher data, respectively. In each case, the test accuracy in the front image was higher than the test accuracy in the back image.

Further, FIG. 80 of FIG. 11 shows the image of the front side of the barley when the neural network using Xception is trained a sufficient number of times by mixing the image of the front side and the image of the back side of the barley which are the teacher data. The judgment result when used as a test data set is shown. The test accuracy of the determination shown in the explanatory diagram 80 was 74%. Further, FIG. 84 shows the determination result when the image of the back side of the barley is used as a test data set in the above case. The test accuracy of the determination shown in the explanatory diagram 84 was 83%.

Further, FIG. 82 shows the determination result when the front side image of barley is used as a test data set when the neural network using Xception is trained a sufficient number of times using only the front side image of barley as teacher data. Shows. Explanatory The test accuracy of the determination shown in FIG. 82 was 63%.

Further, FIG. 86 shows the determination result when the image of the back side of barley is used as a test data set when the neural network using Xception is trained a sufficient number of times using only the image of the back side of barley as teacher data. Shows. The test accuracy of the determination shown in the explanatory diagram 86 was 79%. Of the four images of false detection in the explanatory diagram 86, three images were contaminated particles having a slightly high concentration of 1 to 5 ppm.

As described above, in the results of learning with Xception in FIG. 11, when the front side image and the back side image of barley are mixed in the teacher data, and when only the front side and back side images are used for the teacher data, respectively. In each case, the test accuracy in the image on the back side was higher than the test accuracy in the image on the front side, which was the opposite of the result when learning was performed with AlexNet in FIG. That is, it was suggested that the judgment accuracy can be improved by making a judgment using a suitable neural network on the front side and the back side of the wheat grain.

Further, FIG. 88 of FIG. 12 shows a case where the front side image and the back side image of barley, which are the teacher data, are mixed and the neural network using AlexNet and the neural network using Xception are trained a sufficient number of times. The image on the front side of barley is judged using a neural network using AlexNet, and the image on the back side is judged using a neural network using Xception. Further, the explanatory view 88 corresponds to the total of the determination result shown in the explanatory view 72 and the determination result shown in the explanatory view 84. Explanatory The test accuracy of the determination shown in FIG. 88 was 82%. In addition, one image of false detection in this result was an image of contaminated particles having a slightly high concentration of 1 to 5 ppm.

Further, in the explanatory diagram 90, a neural network using AlexNet is trained a sufficient number of times using only the image on the front side of barley as training data, and a neural network using Xception using only the image on the back side of barley as training data is sufficient. In the case of training the number of times, the judgment result when the front side image of barley is judged using the neural network using AlexNet and the back side image is judged using the neural network using Xception. Shows. Further, the explanatory diagram 90 corresponds to the total of the determination result shown in the explanatory diagram 74 and the determination result shown in the explanatory diagram 86. The test accuracy of the determination shown in the explanatory diagram 90 was 80%. In addition, 3 out of 4 images of false detection in this result were images of contaminated particles having a slightly high concentration of 1 to 5 ppm.

Further, Explanatory FIG. 92 and Explanatory FIG. 94 are diagrams used for comparison with the determination result shown in Explanatory FIG. 88. Explanatory FIG. 92 shows a test data set of the front and back images of barley when the neural network using AlexNet is trained a sufficient number of times by mixing the front side image and the back side image of barley which are teacher data. The judgment result when used as is shown. Further, the explanatory view 92 corresponds to the total of the determination result shown in the explanatory view 72 and the determination result shown in the explanatory view 76, and the explanatory view 56. Explanatory The test accuracy of the determination shown in FIG. 92 was 80%.

Explanatory FIG. 94 shows a test data set of the front and back images of barley when the neural network using Xception is trained a sufficient number of times by mixing the front side image and the back side image of barley which are teacher data. The judgment result when used as is shown. Further, the explanatory view 94 corresponds to the total of the determination result shown in the explanatory view 80 and the determination result shown in the explanatory view 84, and the explanatory view 58. Explanatory The test accuracy of the determination shown in FIG. 94 was 79%.

As mentioned above, it is considered that the results shown in FIG. 12 did not give clear results due to the small number of images (samples) used for learning and testing, but the images on the front side and the back side of barley were judged by different networks. The test accuracy (80 to 82%) when the above was performed showed an improvement tendency as compared with the test accuracy (79 to 80%) when the front side and the back side were not distinguished.

[Example 5]
A fifth embodiment of the present disclosure will be described below. In this embodiment, cross-validation is performed on the barley of the sample series including a part of the sample series used in Example 1 by using the judgment device according to the above-mentioned judgment device 1 in the above embodiment. The results of the results will be explained. FIG. 13 shows information about the barley sample used as the teacher data and test data set of this example.

In FIG. 13, the "sample series number" indicates a number for distinguishing each sample. Although some of the items are different between FIGS. 13 and 5, the barley of the sample series numbers 1 to 3 in FIG. 13 corresponds to the barley of the sample series T1, T2, and U01 of FIG. 5, respectively.

In FIG. 13, "sample series", "cultivar", "Fusarium head blight infection condition", and "grain thickness" have the same meaning as each item in FIG. Further, the meaning of the item indicating the number of wheat grains of each contamination concentration for each of the four columns from the "total number of sample grains" is the same as in FIG. 5, but in FIG. 13, the contamination concentration is different from that in FIG. Is distinguished.

In FIG. 13, the “mycotoxin source concentration estimated from the data for each grain” indicates the mycotoxin concentration of the entire sample series calculated back from the mycotoxin concentration measured for each grain of barley contained in the sample series. ing. Further, the "shooting digital camera model" indicates the model of the digital camera used for imaging the barley of each sample series number. For example, the images of barley of sample series numbers 4 and 5 are images taken by a digital camera Y of the same model. Further, "the number of images on the front and back" indicates the number of images on the front side and the back side of the barley grains used in the learning of this embodiment. At the time of learning, the image for learning was subjected to data augmentation such as position movement, left-right inversion, enlargement / reduction, and color tone change.

The concentration of mycotoxins was measured for the barley of sample series numbers 4 to 6 under different conditions from the barley of sample series numbers 1 to 3. Mycotoxins were measured by ELISA for barley of sample series numbers 1 to 3, and mycotoxins were measured by LC-MS / MS for barley of sample series numbers 4 to 6. Further, in the barley of sample series numbers 4 to 6, the mycotoxin concentration (DON + NIV) is converted into DON by converting DON-3-glucoside, which is a glycoside of DON, which may be measured separately from DON. Calculated.

Further, in this embodiment, it is decided to perform 6-fold cross-validation in which learning is performed using data derived from 5 types of sample series out of 6 types, and verification and testing are performed using data derived from the remaining 1 type of independent sample series. And as verification data, two-class classification was learned using images of high-concentration contaminated grains of barley exceeding 5 ppm and images of non-to low-concentration contaminated grains of barley less than 0.5 ppm. However, as will be described later, as the test data set, assuming the actual use situation of this technology, not only the images of the barley of the high-concentration contaminated grains and the non-low-concentration contaminated grains, but also in 0.5 to 5 ppm. Images of barley with concentration-contaminated grains were also used. Also, during learning, the class with a small number of images amplifies the data by flipping left and right so that the number of images is almost the same between the classification classes in each sample series, or within about 1: 2, and the other class is appropriate. The number of images was reduced and used. However, in the verification data, images of barley less than 0.5 ppm were used without reduction.

14 and 15 show the learning process when cross-validation is performed using the images of barley of the sample series 1 to 6 of FIG. In the cross-validation, a neural network using AlexNet was trained by learning data and verification data which are teacher data.

In FIGS. 14 and 15, graphs 101 to 106 show the accuracy at the time of learning and at the time of verification, respectively. Graphs 111 to 116 correspond to the values of the loss function at the time of learning and at the time of verification. The horizontal axis of each graph is the number of epochs.

Further, in FIGS. 14 and 15, the description such as “23456train_1val” indicates the combination of the sample series in the cross-validation, and corresponds to the “learning_validation data set” in FIG. 16 and the like described later. For example, “23456train_1val” in FIG. 14 indicates that the learning was performed using the images of the sample series numbers 2 to 6 as training data and the images of the sample series number 1 as verification data. Further, "13456 train_2val" indicates that the learning was performed using the images of the sample series numbers 1 and 3 to 6 as training data and the images of the sample series number 2 as verification data. Also, other combinations of sample series in cross-validation will be described in the same manner.

FIG. 16 shows the results of the cross-validation shown in FIGS. 14 and 15. The details will be described later, but the accuracy, precision, recall, F1 value, etc. corresponding to each combination of the sample series are listed by rounding the values. In FIG. 16, “TP” (True Positive) indicates the number of images determined to be highly contaminated particles by the determination device with respect to the image of barley which is actually a highly contaminated grain. "FP" (False Positive) indicates the number of images determined by the determination device to be highly contaminated grains with respect to the image of barley which is not actually a highly contaminated grain. "FN" (False Negative) indicates the number of images determined by the determination device to be not high-concentration contaminated grains with respect to the image of barley which is actually a high-concentration contaminated grain. "TN" (True Negative) indicates the number of images determined by the determination device to be not highly contaminated grains with respect to the image of barley that is not actually highly contaminated grains. The larger the value of TP and TN is, the more desirable it is, and the smaller the value of FP and FN is, the more desirable it is.

"Accuracy" indicates the ratio at which the determination device could correctly determine whether or not the target is a highly concentrated contaminated grain. The value of "accuracy" is obtained by the formula (TP + TN) / (TP + FP + FN + TN). The "adaptation rate" indicates the ratio of the images that were actually high-concentration contaminated grains to the images that the determination device determined that the target was high-concentration contaminated grains. The value of the "compliance rate" is obtained by the formula (TP) / (TP + FP). The "reproducibility" indicates the ratio of the images that are actually high-concentration contaminated particles to the image that the determination device can determine as high-concentration contaminated particles. The value of "recall rate" is obtained by the formula (TP) / (TP + FN). The "F1 value" is a value indicating a harmonic mean of the precision rate and the recall rate. The "F1 value" is obtained by the formula (2 x conformance rate x recall rate) / (compliance rate + recall rate).

FIGS. 17 and 18 show determination results and the like when an image as a test data set is given as an input to the trained neural network trained in the above-mentioned cross-validation. Here, as a test data set, all grains in each sample series, namely high-concentration contaminated grains of barley above 5 ppm, medium-concentration contaminated grains of 0.5-5 ppm, and non-low of barley less than 0.5 ppm. Two classes of whether or not the particles were highly contaminated were classified using one front and one back of the image of the highly contaminated particles.

In FIG. 17, “number of test images” indicates the number of images of barley used in the test data set. Further, the barley sample series number used for the image of the test data set here is the same as the barley sample series number used for the verification data. For example, an image of barley of sample series number 1 is input as a test data set in the trained neural network corresponding to the row in which the "training_verification data set" is "23456train_1val". In addition, the test data set includes one front side image and one back side image of each barleycorn. That is, for example, the number of test images 108 corresponds to 54 grains of barley. Further, in the image of the test data set, data augmentation such as position movement and left-right inversion is not performed. Further, some items such as the "number of test images" are also described in FIG.

In addition, the "mycotoxin concentration estimated from the data for each grain" is calculated back from the mycotoxin concentration measured for each grain of barley in the sample series used for the verification and test data set, and is calculated for the entire sample series. It shows the concentration of mycotoxins. This value is calculated directly from the data of the grains (44 to 60 grains) of each sample series, and is the mold poison concentration (Fig. 5 and Fig. 6) of the sample fraction of each sampling source of these samples. Does not always match due to sampling error. The "probability threshold for determining a high-concentration grain" is, if the probability that the target grain image is estimated to be a high-concentration contaminated grain is 0% or more, whether the grain image is determined to be a high-concentration contaminated grain. Indicates the threshold value of. For example, when the value of the probability threshold is 0.5, the determination device contaminates the grain image with a high concentration when it is estimated that the target grain image is a highly contaminated grain with a probability of 50% or more. Judged as a grain. "Number of high-concentration grain determination images" indicates the number of images determined by the determination device to be images of high-concentration contaminated grains. "Concentration after sorting and removal" is the mold of the entire remaining grain image when the image determined to be highly contaminated grains is excluded from the images (test data set) of all grains of the sample series used for the verification data. It shows the poison concentration. Here, the "sorting yield" corresponds to the entire grain image before sorting, which corresponds to the entire remaining grain image when the determination device removes the grain image determined to be highly contaminated grains. The ratio (weight ratio) to the grain weight is shown. The "mycotoxin reduction rate" is the grain image with the original mycotoxin concentration when the grain image determined to be highly contaminated grains is excluded from the entire grain image (test data set) of the sample series used for the verification data. It shows what percentage of the total was reduced. Further, the "average when the sorting yield is about 80%" in the last row shows the average value of each item when the sorting yield of about 80% shown in the underlined portion is used. In the results shown in FIG. 17, although there are differences depending on the combination of the cross-validation data sets, the effect of reducing the mycotoxin concentration by about 50% on average was confirmed when the sorting yield was set to about 80%.

FIG. 18 shows the mold for each grain of the grain image determined to be highly contaminated grains in the test data set of the sample series used for the verification data and the remaining grain images excluding the grain image of the highly contaminated grains. Includes an item for the significance test of poison concentration. In FIG. 18, “average concentration of high-concentration determined grains per grain” is the average value of the mycotoxin concentration of each grain of the grain image determined to be highly contaminated grains in the test data set of the sample series used for the verification data. Shows. "Average concentration per grain after removal of high-concentration judgment grains" is the mold of each grain of the remaining grain images excluding the grain images judged to be highly contaminated grains in the test data set of the sample series used for the verification data. It shows the average value of the poison concentration. "***" in the "significance test" indicates that there is a significant difference at the significance level of 0.1%, and "**" indicates that there is a significant difference at the significance level of 1%. ing. Further, "*" indicates that there is a significant difference at the significance level of 5%, and "ns" indicates that there is no significant difference at the significance level of 5%.

In barley, there has been no sorting method effective for reducing mycotoxins after harvesting other than grain thickness sorting and specific gravity sorting, but according to the determination device of this embodiment, after grain thickness sorting (here, 2.6). It has been suggested that high-concentration contaminated grains that are difficult to distinguish with the naked eye are removed from barley samples contaminated with mycotoxins (mm or more) to reduce the concentration of mycotoxins. As described above, in this embodiment, the sample is derived from the sample independent of the training data even though the number of grains of the sample used for the teacher data is relatively small and the image taken by a general digital camera is used. In the verification of the grain sorting effect using a test data set containing images of grains of various degrees of contamination, the effect beyond imagination was obtained. In this example, barley was targeted, but mycotoxins may be accumulated even in wheat grains having a healthy appearance or unclear damage, and mycotoxins-contaminated wheat samples containing such contaminated grains may also accumulate mycotoxins. It is expected that the mycotoxin reduction effect by grain sorting can be improved by using the determination device according to this embodiment. It is considered that the judgment performance of the judgment device will be improved by increasing the learning data, examining the wavelength range and the like constituting the image, and examining and improving the machine learning model.

Further, FIG. 19 shows the relationship between the probability threshold for determining high-concentration contaminated particles and the sorting yield for the test data set of the combination of the above-mentioned sample series. The sorting yield is a value in terms of the number of grains (images) here, but in a barley sample having a certain grain thickness, this value can be considered as an approximate value of the sorting yield in terms of weight ratio. can. In FIG. 19, the 80% sorting yield line described with reference to FIGS. 17 and 18 is shown.

Further, in FIG. 19, for example, in the combination of the sample series of "13456train_2val" and "12456train_3val", the barley of the sample series No. 2 and the barley of the sample series No. 3 which are the test data sets have a probability threshold value of 0.5 (50). Even in the case of (corresponding to%), it indicates that the sorting yield is 80% or more. On the other hand, for example, in the barley of sample series No. 4 and the barley of sample series No. 6, in order to make the sorting yield 80% or more, the probability threshold value may be 0.8 (80%) or more. You will need it.

Further, the relationship between the above-mentioned probability threshold value and the sorting yield (grain (image) number ratio or weight ratio) can be calculated or estimated by the determination device with reference to the barley grain image of each sample series. As an example, the sorting yield (ratio of grain (image) number) (%) is (1- (the number of images determined by the determination device to be an image of highly contaminated grains / the number of images of barleycorn in the target sample series). )) Calculated by the formula of × 100. In addition, as teacher data, the relative grain weight data corresponding to each grain image, or the average grain weight of low-concentration contaminated grains or uncontaminated grains less than a predetermined reference value in the same sample lot of the grain image is used as a reference. By learning the grain weight ratio data as a set with each grain image, it is possible to estimate each grain weight or relative grain weight ratio from each grain image to be processed. From this, using the estimated value, the sorting yield (weight ratio) (%) is set to (1- (total grain weight (estimated value) of each image determined by the determination device as highly contaminated grains / target). Total of grain weights (estimated values) of all images of barleycorn in the sample series) x 100 and (1- (total of relative grain weight ratios (estimated values) of each image judged by the judgment device as highly contaminated grains) Value / Total value of relative grain weight ratio (estimated value) of all images of barleycorn of the target sample series)) may be calculated (estimated) by the formula of 100. Alternatively, as described above, under predetermined conditions, the sorting yield based on the grain (image) number ratio and the sorting yield based on the weight ratio can be treated as approximate values to each other.

Thereby, for example, the probability threshold and the sorting yield estimated to be high-concentration contaminated grains by referring to the image of a certain amount of grains extracted from the barley sample lot to be sorted and removed in advance by this determination device. By obtaining information on the relationship with (grain ratio or weight ratio), the probability threshold to be used for determining highly concentrated contaminated grains in the barley sample lot should be determined using the corresponding sorting yield value as a criterion. Can be done.

At this time, if necessary, information on the relationship between the probability threshold and the sorting yield is obtained separately according to the orientation of the grains such as the front side and the back side, and different probability thresholds are contaminated at high concentration according to the orientation of the grains. It may be used for grain determination.

In addition, by accumulating data such as grain-by-grain images and mycotoxins concentration of various sample series obtained under various conditions, the judgment performance of the judgment device will be improved by using them for learning, and the figure will be shown. The data on the relationship between the probability threshold and the sorting yield corresponding to various sample series as shown in 19, the corresponding mycotoxin reduction rate, and the data on the mycotoxin concentration after removal of high-concentration contaminated particles are also updated. While accumulating. By developing an estimation model using these data, the image of a certain amount of grains extracted from the sample lot for the barley sample to be newly judged can be referred to by the judgment device (at this time, the relevant sample lot). Information on the relationship between the probability threshold and the sorting yield in the determination device), the original mycotoxin concentration or degree of contamination of the sample lot, as well as the selected probability threshold and the mycotoxin reduction corresponding to the sorting yield. It is possible to estimate the rate, mycotoxin concentration or degree of contamination after removal of high-concentration contaminated particles. This information can be used as a judgment material when determining the probability threshold value and the sorting yield to be used for determining the highly concentrated contaminated grains of the barley sample lot. In the development of the estimation model, for example, methods such as a generalized linear model, Bayesian estimation, and machine learning can be used. In addition, even for samples for which the mycotoxin data for each grain is not always available, the estimated model is based on the mycotoxin concentration before sorting contaminated grains and the data of the mycotoxin concentration after sorting to which a predetermined probability threshold and sorting yield are applied. It can be used for the development and improvement of. In addition, various incidental information of each sample, for example, information on the type of target crop (for example, barley, Nijo barley, six-row barley, bare barley, etc.), varieties, various cultivation conditions (cultivation area, weather conditions, dominant bacterial species, etc.) Information (data) such as information and control conditions) can also be used for the development and improvement of the estimation model.

In Example 5, a neural network classification model is used as the determination device, but when a regression model is used, the “probability threshold”, which is the determination threshold for mycotoxins with high concentration of mycotoxins, in this example is It can be replaced with "estimated mycotoxin concentration". It is also possible to use other appropriate determination thresholds depending on the type of machine learning model used in the determination device. It should be noted that a machine learning model in which there is no judgment threshold for determining whether or not the particles are highly contaminated with mycotoxins is not assumed. ) In the classification model to be returned, the reference value as "mycotoxins highly contaminated grains", that is, the lower limit of the mycotoxins concentration or the degree of contamination of the mycotoxins highly contaminated grains used in the teacher data of the model ( For example, 5 ppm) can be considered to be the determination threshold in the model. As described above, in many cases, the reference value of "mycotoxin high-concentration contaminated particles" can be regarded as the determination threshold value as it is.

1 Judgment device (learning device)
10 Control unit 12 Acquisition unit 14 Judgment unit 16 Learning unit 20 Storage unit 22 Action unit 24 Imaging unit 26 Measurement unit

Claims (15)

1. An acquisition unit that acquires an image containing one or more grains of granular agricultural products,
   Using a trained machine learning model in which the image is input and the determination result regarding the mycotoxin concentration or the degree of contamination in the grains of the granular agricultural product is output for each grain, (1) one or more grains of the granular agricultural product are individually output. Determine if the grain or whole of the crop is contaminated with mycotoxins above a predetermined reference value or in a predetermined concentration range, or (2) the mycotoxin concentration of one or more individual grains or the whole of the granular agricultural product. Alternatively, a judgment unit that determines the degree of contamination and
   A determination device characterized by comprising.
2. The acquisition unit acquires an image containing one or more grains of granular agricultural products imaged in the ultraviolet or infrared region.
   The determination unit is a machine learning model into which the image is input, and is a machine learning model learned by using an image containing one or more grains of granular agricultural products captured in the ultraviolet or infrared region as teacher data. The determination device according to claim 1, wherein the determination is performed using the determination device.
3. The determination device according to claim 1 or 2, wherein the acquisition unit acquires an image containing one or more grains of barley.
4. The determination unit
   After determining the orientation of the grains of the granular agricultural products contained in the image,
   The determination device according to any one of claims 1 to 3, wherein the determination is performed using a trained machine learning model according to the orientation of the grains of the granular agricultural product contained in the image.
5. The acquisition unit acquires an image containing one or more grains of the harvested granular agricultural product.
   An action part that aligns the grains of the target granular agricultural products in a predetermined direction,
   The determination according to any one of claims 1 to 4, wherein the action unit captures an image of grains of granular agricultural products in which the directions are aligned, and further comprises an imaging unit that supplies the acquisition unit. Device.
6. The determination unit
   It is determined whether or not each of the individual grains of the granular agricultural product contained in the input image is contaminated with mycotoxins above a predetermined reference value using a predetermined determination threshold value.
   The selection yield, which is the ratio of the number of grains determined not to be contaminated with mycotoxins above the predetermined reference value, among the set of grains subject to the determination, is set to one or more of the determination thresholds. The determination device according to any one of claims 1 to 5, further comprising a function of calculating or estimating for each.
7. The determination unit
   It is determined whether or not each of the individual grains of the granular agricultural product contained in the input image is contaminated with mycotoxins above a predetermined reference value using a predetermined determination threshold value.
   Among the set of grains to be determined, the selection yield, which is the ratio of the weight of the grains determined not to be contaminated with mycotoxins above the predetermined reference value, is set to one or more of the determination thresholds. The determination device according to any one of claims 1 to 5, further comprising a function of calculating or estimating for each.
8. The determination unit
   It is determined whether or not each of the individual grains of the granular agricultural product contained in the input image is contaminated with mycotoxins above a predetermined reference value using a predetermined determination threshold value.
   When the grains determined to be contaminated with mycotoxins above the predetermined reference value are removed from the set of grains to be determined, the mycotoxin reduction rate or the total set of remaining grains is mycotoxin. The determination device according to any one of claims 1 to 7, further comprising a function of estimating a concentration or a degree of contamination for each of one or more of the determination thresholds or selection yields.
9. An acquisition unit that acquires an image containing one or more grains of the granular agricultural product and concentration information indicating the mycotoxin concentration or the degree of contamination of each grain of the granular agricultural product.
   The image and the machine learning model acquired by the acquisition unit are obtained by inputting an image containing one or more grains of the granular agricultural product and outputting the determination result regarding the mold poison concentration or the degree of contamination in the grains of the granular agricultural product for each grain. A learning device including a learning unit for learning using a set of concentration information as teacher data.
10. The learning device according to claim 9, further comprising a measuring unit that measures or estimates the mycotoxin concentration or the degree of contamination of the grains of the granular agricultural product and supplies the result to the acquisition unit.
11. The learning unit
    An image containing grains of first-class granular agricultural products whose mycotoxin concentration or degree of contamination is equal to or higher than a predetermined standard value, and whether the mycotoxin concentration or degree of contamination is equal to or less than a predetermined threshold set below the predetermined standard value. Alternatively, an image containing grains of a second class of granular agricultural product that is non-contaminated is input to the machine learning model.
    The machine learning model is trained to output a determination result as a probability value of whether or not the grain of the granular agricultural product contained in the input image is a grain contaminated with mycotoxins having a predetermined reference value or more. The learning device according to claim 9 or 10.
12. The acquisition step of acquiring an image containing one or more grains of granular agricultural products,
    Using a trained machine learning model in which the image is input and the determination result regarding the mycotoxin concentration or the degree of contamination in the grains of the granular agricultural product is output for each grain, (1) one or more grains of the granular agricultural product are individually output. Determine if the grain or whole of the crop is contaminated with mycotoxins above a predetermined reference value or in a predetermined concentration range, or (2) the mycotoxin concentration of one or more individual grains or the whole of the granular agricultural product. Alternatively, a determination step for determining the degree of contamination and
    A determination method characterized by including.
13. An acquisition step for acquiring an image containing one or more grains of the granular agricultural product and concentration information indicating the mycotoxin concentration or the degree of contamination of each grain of the granular agricultural product.
    A machine learning model in which an image containing one or more grains of a granular agricultural product is input and a determination result regarding the mold poison concentration or the degree of contamination in the grains of the granular agricultural product is output for each grain is obtained with the image and the image acquired in the acquisition step. A learning method including a learning step in which a set of density information is used as teacher data for learning.
14. The control program for operating the computer as the determination device according to claim 1, and the control program for operating the computer as the acquisition unit and the determination unit.
15. The control program for operating the computer as the learning device according to claim 9, and the control program for operating the computer as the acquisition unit and the learning unit.

Images