JP2024027765A - Quality determination method for fresh food and quality determination system therefor - Google Patents

Abstract

To provide a quality determination technology for fresh foods, capable of determining the quality of fresh foods of any type, such as seafood, meat and fruit under an environment of visible light by using a biochemistry index.SOLUTION: A quality determination method for fresh foods is designed for determining quality of the fresh foods from image data of the fresh foods, the method including an image/biochemistry index database construction step of constructing a database from image data for learning obtained by imaging the fresh foods and biochemistry indices obtained by measuring the fresh foods for learning, a determination model learning step of learning a determination model by using a machine learning from the image data of the database to extract a color, a reflected light amount and a feature amount expressing a surface shape and associating them with the biochemistry indices, and a fresh food quality determination step of determining quality of the fresh foods to be determined in quality by estimating the closest determination model from the image data for determination obtained by imaging the fresh foods to be determined in quality.SELECTED DRAWING: Figure 2

Description

The present invention relates to a fresh food quality determination technique for determining the quality of fresh food from image data of the fresh food.

In recent years, in the distribution and sale of fresh foods, there has been a demand for higher freshness and higher quality. For example, this is particularly important for seafood. Fish and shellfish undergo significant changes after death and their freshness quickly deteriorates, so in order to build a system that can provide highly fresh products, it is necessary to have technology to determine whether quality maintenance, including freshness, is being maintained. By implementing this technology in society, it will be possible to easily determine the quality of seafood products at wholesale markets in production and consumption areas and during transportation, contributing to the branding and high quality of seafood products. Can be done.

Currently, the following are the basic indicators of quality, including freshness. On the one hand, changes in appearance, odor, and color are determined by sensory evaluation methods. On the other hand, the composition of ATP-related compounds (K value), volatile basic nitrogen (VBN), pH value, muscle pigment, etc. are determined by chemical component analysis methods. Here, the sensory evaluation method requires a comprehensive judgment based on changes in the appearance, smell, and color of seafood, and there are large individual differences. In addition, chemical component analysis methods require specialized analysis techniques, take time to measure and analyze, making it difficult to make a determination in a short time, and the equipment is expensive, so it is difficult to install it in markets, stores, homes, etc. Sometimes it is difficult.

In order to solve these problems, several freshness determination techniques are known. For example, a skilled craftsman accumulates data on tuna connoisseurs, and uses image analysis technology to classify tuna into three categories: top-quality, high-quality, and average. There are things to judge by type. However, such techniques for determining the freshness of seafood are not as accurate as the sensory evaluation by craftsmen, and therefore cannot meet the demand for more precision and higher quality in the storage of seafood.

In addition, another technology utilizes artificial intelligence to evaluate the freshness of whitebait fish. After taking images of landed whitebait fish, discerning fishermen label them by grade, focusing on the transparency. The data is stored, and the freshness is judged in four stages. Images of whitebait fish to be evaluated for freshness are compared with accumulated data to determine freshness, but this method is also not as accurate as sensory evaluation by fishermen.

In addition, the catch identification system using artificial intelligence in Patent Document 1 includes a fish species feature storage unit that stores fish species features for each fish species of the catch, and a fish species feature storage unit that images the catches individually to extract the features. an imaging section, a fish species determination section that compares the feature amount with the fish species feature amount and determines the fish species of the catch, an identification number assigning section that assigns an identification number to the catch, and a It is equipped with an individual feature amount extraction unit that extracts individual feature amounts that can be identified as catches, and an individual catch database that stores the individual feature amounts by linking them to identification numbers for each catch. However, the artificial intelligence-based catch identification system disclosed in Patent Document 1 only determines the species of fish, and cannot determine the freshness of the fish itself.

Further, the fish freshness estimation method of Patent Document 2 includes an absorbance spectrum acquisition step of acquiring an absorbance spectrum obtained by irradiating the fish's eyes with light including all or a part of the wavelength band from 315 nm to 450 nm, and the acquired absorbance. The freshness of fish is estimated using the shape of the spectrum. However, visible light cannot be used in the wavelength band of 315 nm to 450 nm, and freshness cannot be estimated without a fisheye.

Patent No. 6401411 Japanese Patent Application Publication No. 2015-232543

In view of the above points, the present invention provides a quality determination method for fresh foods that can determine the quality of fresh foods regardless of type, such as seafood, meat, fruits and vegetables, using biochemical indicators in a visible light environment. The challenge is to provide technology.

[1] A fresh food quality determination method for determining the quality of the fresh food from image data of the fresh food, the method comprising:
an image/biochemical index database construction step of constructing a database from learning image data obtained by photographing the fresh food for learning and biochemical indices obtained by measuring the fresh food for learning;
a judgment model learning step of extracting feature quantities expressing color, amount of reflected light, and surface shape from the image data as training data of the database using machine learning, and learning a judgment model by associating them with biochemical indicators;
A fresh food quality determination step of determining the quality of the fresh food that is the subject of quality determination by estimating the closest determination model from the image data for determination taken of the fresh food that is the subject of quality determination. have,
The image/biochemical index database construction step includes:
Prepare the fresh food after landing or 0 days after harvesting, and create the image data for learning by photographing with a camera with illumination of a predetermined color temperature applied to the fresh food for learning, Next, the process of rotating the fresh food for learning by a predetermined angle in planar view and photographing it with the camera to create the image data for learning is repeated n times;
Further, changing the color temperature by a predetermined value and photographing with the camera to create the image data for learning is repeated m times,
Further, the process of photographing the 0-day-old fresh food with the camera every predetermined time period to create the image data for learning is repeated p times, and the image data for learning is created for a total of n×m×p times. The method is characterized by comprising an image data creation sub-step of creating the image data.

According to this configuration, there is an image/biochemical index database construction step of constructing a database from learning image data obtained by photographing fresh food for learning and biochemical indices obtained by measuring fresh food for learning. Therefore, rather than using the intuition of a connoisseur, the quality of fresh foods, including their freshness, can be determined based on the biochemical index values obtained by measuring fresh foods and image data. The judgment model learning process uses machine learning to extract features expressing color, amount of reflected light, and surface shape from image data as training data in the database, and learns a judgment model by linking them to biochemical indicators. The process includes a fresh food quality determination process in which the quality of the fresh food subject to quality determination is determined by estimating the closest determination model from the image data for determination obtained by photographing the target fresh food. The quality of fresh food can be determined instantly using artificial intelligence.

Furthermore, in the present invention, in the image/biochemical index database construction step, fresh food is prepared 0 days after landing or harvesting, and a camera is used with lighting of a predetermined color temperature shining on the fresh food for learning. to create image data for learning by photographing the fresh food for learning, then rotate the fresh food for learning by a predetermined angle in planar view and photograph it with a camera to create image data for learning, repeating this process n times. Furthermore, the process of changing the color temperature by a predetermined value and photographing with a camera to create image data for learning is repeated m times.Furthermore, fresh food that is 0 days old is photographed with a camera every predetermined time period. The image data creation sub-step includes repeating p times of creating image data for learning using the image data, and creating image data for learning for a total of n×m×p times. For this reason, the judgment model is learned from a large amount of training image data consisting of different color temperatures and different angles, so it is not affected by the brightness or darkness of the environment or shooting angle in which the fresh food that is subject to quality judgment is photographed. , quality can be determined accurately. In this way, in a visible light environment, the quality of fresh foods, regardless of type, such as seafood, meat, fruits and vegetables, can be determined using biochemical indicators.

[2] Preferably, the image/biochemical index database construction step includes:
The biochemical indicators include K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid, and these biochemical indicators are measured. A biochemical index measurement sub-step obtained by measuring with a device,
a database construction sub-step of constructing a database from the image data for learning in the image data creation sub-step and the biochemical index in the biochemical index measurement sub-step;
The process ends when the fresh food for learning exceeds a predetermined storage period.

According to this configuration, biochemical indicators include the K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, and muscle pigment, and these biochemical indicators are measured in the biochemical indicator measurement sub-step. Since the chemical index is obtained by measuring it with a measuring device, there are no individual differences unlike in sensory tests, and no matter who is judging the fresh food, the same judgment result can be obtained, making quality judgment stable and highly accurate. It can be done.

[3] Preferably, the judgment model learning step includes:
an image data input sub-step of inputting the image data of the database to an analysis section;
a biochemical index input sub-step of inputting the measured biochemical index to the analysis section;
The analysis unit uses machine learning to extract feature values expressing color, amount of reflected light, and surface shape, and creates a judgment model by matching them with the elapsed time from day 0 of the fresh food for learning. and a decision model learning sub-step for learning the decision model.

According to this configuration, in the judgment model learning sub-step, the analysis section uses machine learning to extract feature quantities expressing the color, amount of reflected light, and surface shape, and extracts features representing the color, amount of reflected light, and surface shape, Since the judgment model is learned in correspondence with the elapsed time, it is also possible to judge the quality of fresh food according to the number of days elapsed.

[4] Preferably, the fresh food quality determination step includes:
a quality determination image data input sub-step of inputting the image data of the quality determination target obtained by photographing the fresh food target of quality determination with a camera to the analysis unit;
a determination model estimation sub-step in which the analysis unit extracts a feature amount from the inputted image data to be determined for quality, and estimates the determination model having the closest feature amount from the database;
The method further includes a biochemical index output sub-step of outputting a biochemical index of the fresh food subject to quality determination using the biochemical index of the determination model estimated in the determination model estimation sub-step.

According to this configuration, when the analysis unit extracts feature quantities from inputted image data to be judged for quality by machine learning, a deep learning method using a so-called neural network or a patent application, which is also an invention of the present inventor, is used. 2022-52171, the learning quality of the judgment model can be improved and the quality of fresh food can be improved. The accuracy of determination can be improved.

[5] Preferably, the image/biochemical index database construction step includes:
Prepare the fresh food that has been freshly harvested for 0 days, and photograph the fresh food for learning with a camera while shining a light of a predetermined color temperature on the fresh food for learning to create the image data for learning, and then Repeating 32 times to create the image data for learning by rotating the fresh food for learning by 360°/32 in plan view and photographing it with the camera;
Further, the color temperature is divided into seven stages from 3300K to 5600K, and each color temperature is photographed with the camera to create the image data for learning, and this is repeated seven times,
Furthermore, the above-mentioned fresh food that is 0 days old is photographed with the above-mentioned camera every 12 hours to create the above-mentioned image data for learning, which is repeated 14 times to create the above-mentioned image data for 7 days worth of learning. do.

According to this configuration, the color temperature is divided into seven levels from 3,300K to 5,600K, and the fresh food for learning is rotated by 360°/32 in planar view for each color temperature and photographed with a camera. Since the process of creating learning image data is repeated 32 times, it is possible to improve the learning quality of the determination model and improve the accuracy of fresh food quality determination.

[6] Preferably, a fresh food quality determination system that determines the quality of the fresh food from image data of the fresh food,
Equipped with a teacher data creation section, a biochemical index measurement section, an analysis section, and a quality judgment target shooting display section,
The teacher data creation unit includes a light that illuminates the fresh food with a changeable color temperature, a camera that photographs the fresh food, and a rotating device that is rotatable at a predetermined angle in a plan view and on which the fresh food is placed. Equipped with a stand,
The image data for learning is created by photographing the fresh food for learning with the camera while illumination with a predetermined color temperature is applied to the fresh food for learning, and then the fresh food for learning is viewed from a predetermined angle in plan view. Repeating n times to rotate the image data by the same amount and take a picture with the camera to create the image data for learning,
Further, changing the color temperature by a predetermined value and photographing with the camera to create the image data for learning is repeated m times,
Further, the process of photographing the 0-day-old fresh food with the camera every predetermined time period to create the image data for learning is repeated p times, and the image data for learning is created for a total of n×m×p times. The image data is created;
The biochemical index measurement unit is a measuring device that measures biochemical indexes including at least K value, which is the composition of ATP-related compounds, VBN, which is volatile base nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid. and
The analysis unit includes a database configured to include the image data for learning and the biochemical index corresponding to these image data,
A determination model is learned by extracting feature quantities expressing color, amount of reflected light, and surface shape using machine learning from the image data as training data of the database,
The quality determination target photographing display unit uses image data for determination obtained by photographing the fresh food to be subjected to quality determination, and the analysis unit estimates the determination model that is closest to the image data for determination. The present invention is characterized in that the quality of the fresh food subject to quality determination is determined and the biochemical index is displayed.

According to this configuration, the teacher data creation section, the biochemical index measurement section, the analysis section, and the quality judgment target photographing and display section prepare the fresh food on the 0th day after landing or harvesting, and adjust the color temperature to a predetermined color temperature. Create image data for learning by shining a light on the fresh food for learning with a camera, then rotate the fresh food for learning by a predetermined angle in a planar view and photograph it with the camera. Repeat n times to create image data for learning with By repeating p times of photographing fresh food for eyes with a camera every predetermined time period and creating image data for learning, it is possible to create image data for learning for a total of n×m×p times. For this reason, the judgment model is learned from a large amount of training image data consisting of different color temperatures and different angles, so it is not affected by the brightness or darkness of the environment or shooting angle in which the fresh food that is subject to quality judgment is photographed. , quality can be determined accurately. In this way, in a visible light environment, the quality of fresh foods, regardless of type, such as seafood, meat, fruits and vegetables, can be determined using biochemical indicators.

[7] Preferably, the illumination can be irradiated with the color temperature changed in seven stages from 3300K to 5600K,
The rotary table is rotatable by 360°/32 in a plan view.

According to this configuration, the color temperature is divided into seven levels from 3,300K to 5,600K, and the fresh food for learning is rotated by 360°/32 in planar view for each color temperature and photographed with a camera. Since the process of creating learning image data is repeated 32 times, it is possible to improve the learning quality of the determination model and improve the accuracy of fresh food quality determination.

It is possible to provide a technology for determining the quality of fresh foods, which can determine the quality of fresh foods regardless of type, such as seafood, meat, fruits and vegetables, using biochemical indicators in a visible light environment.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating an overview of a fresh food quality determination method and a quality determination system of the present invention. It is a figure explaining the main flow of the quality determination method of fresh food. FIG. 2 is a flow diagram illustrating sub-steps of an image/biochemical index database construction step. FIG. 3 is a diagram illustrating a teacher data creation section. FIG. 3 is a diagram illustrating learning image data taken in the image/biochemical index database construction process. FIG. 3 is a diagram illustrating a biochemical index measurement sub-step. This is an example of an image of a raw tuna fillet on the 0th day of landing. This is an example of an image of a raw tuna fillet on the second day of landing. This is an example of an image of a raw tuna fillet on the fourth day of landing. This is an example of an image of a frozen tuna fillet on the 0th day of landing. This is an example of an image of a frozen tuna fillet on the second day of landing. This is an example of an image of a frozen tuna fillet on the fourth day of landing. It is a flow diagram explaining the sub-process of a judgment model learning process. It is a figure explaining a judgment model learning process. It is a flowchart explaining the sub-steps of the fresh food quality determination process. It is a figure explaining the result of the test of the judgment model which made fresh food yellowtail. It is a figure explaining the result of the test of the judgment model using squid as fresh food. It is a figure explaining the result of the test of the judgment model using tuna as fresh food.

Embodiments of the present invention will be described below based on the accompanying drawings. Note that the drawings conceptually (schematically) show each device of the fresh food quality determination system.

As shown in FIGS. 1, 4, and 6, the fresh food quality determination system 10 determines the quality of the fresh food 50 based on the image data 51 (see FIGS. 5, 7 to 12, and 14) of the fresh food 50. The system includes a teacher data creation section 20, a biochemical index measurement section (biochemical index) 40, an analysis section 30, and a quality determination target imaging display section (not shown).

Next, the hardware aspects of the teacher data creation section 20 will be explained.
The teacher data creation unit 20 includes a box-shaped casing 21 with an open front, a door 24 provided with a transparent window that allows the opening of the casing 21 to be opened and closed and allows the interior to be seen through, and a color temperature A lighting (light source) 22 that illuminates the fresh food 50 by making it changeable, a camera 25 that photographs the fresh food 50, and a rotary table 23 that is rotatable at a predetermined angle in plan view and on which the fresh food 50 is placed. We are prepared. The illumination 22 can emit light while changing the color temperature in seven stages from 3300K to 5600K. The rotary table 23 is equipped with, for example, a stepping motor and can rotate by 360°/32 in a plan view.

In addition, the function is to create image data 51 for learning by photographing the fresh food 50 for learning with the camera 25 while shining the illumination 22 of a predetermined color temperature on the fresh food 50 for learning. The process of rotating the image by a predetermined angle in planar view and photographing it with the camera 25 to create learning image data 51 is repeated n times. Further, the process of changing the color temperature by a predetermined value and photographing with the camera 25 to create learning image data 51 is repeated m times. Furthermore, the process of photographing the freshly harvested 0-day-old fresh food 50 with the camera 25 every predetermined period of time to create learning image data 51 is repeated p times, and a total of n×m×p training images are captured. Image data 51 is created.

Next, the biochemical index measuring section 40 will be explained.
The biochemical index measurement unit 40 measures biochemical indexes including at least K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, and muscle pigment, chlorophyll, carotenoid, and ascorbic acid. It is a device. Note that the biochemical index measuring unit 40 may be defined not only as the measuring device itself, but also as each measured value of a biochemical index obtained by measurement by the measuring device.

Next, the analysis section 30 will be explained.
The analysis unit 30 includes a database configured to include image data 51 for learning and biochemical indicators 40 that are input and stored as a set corresponding to these image data 51. The analysis unit 30 is also equipped with artificial intelligence (AI), which uses machine learning to determine the color, amount of reflected light, and surface of the fresh food 50 from the image data 51 as training data in the database. This method extracts feature quantities that express shape and associates them with biochemical indicators to learn a judgment model.

The artificial intelligence of the analysis unit 30 may be of general type. Generally speaking, machine learning refers to technology that learns regularities and judgment criteria from data, predicts and judges unknown things based on the learned regularities and judgment criteria, and analysis technology related to artificial intelligence. Deep learning (deep learning method), which is included in machine learning, may also be used. Deep learning is an extension of neural network analysis method, which is a more basic and wide-ranging machine learning method, and is a highly advanced method. This is a method that enables accuracy analysis and utilization. In one embodiment of the present invention, so-called supervised learning, which is machine learning in which teacher data corresponding to correct answers is given, is used. Note that analysis methods for supervised learning include regression analysis and decision trees.

Neural networks are also applied to the field of image recognition, and a neural network with two or more intermediate layers (hidden layers) is called deep learning. Deep learning automatically extracts the features that characterize the data and learns the rules and regularities contained in the data without the need for humans to tell you where to focus on the given data or specify specific features. This is a method to do so.

The artificial intelligence of the analysis unit 30 of the present invention is not limited to the general one, but is also based on the ``Statistical Distance Matrix Calculation Method'' disclosed in Japanese Patent Application No. 2022-52171, which is also an invention of the present inventor. A deep learning method based on "Visualization method, device, and program" may be used. The accuracy of deep learning can be improved by optimizing the structure of deep learning convolutional layers using statistical distance matrices.

The quality determination target shooting display section (not shown) will be explained.
The quality determination target photographing display unit uses the image data for determination obtained by photographing the fresh food 50 that is the target of quality determination, and uses the analysis unit 30 to estimate the determination model closest to the image data for determination, thereby determining the quality. It determines the quality of target fresh foods and displays biochemical indicators.

The shooting display unit to be judged for quality may be one connected to a PC using the teacher data creation unit 20, but it may also be a combination of an external camera different from the teacher data creation unit 20 and a PC, or even a general It may be a smartphone or tablet. The fresh food 50 that is subject to quality determination is connected to the fresh food quality room determination system 10 (analysis section 30) via a network such as the Internet, and sent from a smartphone to the fresh food quality room determination system 10 (analysis section 30). It is also possible to transmit the image data for determination taken by photographing and return the determination result by the analysis unit 30 to the smartphone and display the determination result.

Further, the environment in which the fresh food 50 to be subjected to quality determination is photographed may be, for example, a living room of an arbitrary facility, the inside of a supermarket, or an environment exposed to visible light that ordinary people usually live in. The present invention uses training data from multiple color temperatures and multiple shooting directions (angles) at the time of the image/biochemical index database construction step and the determination model learning step (details will be described later) to create a determination model from training data. Since the determination model is learned based on the obtained information, the photographing environment of the fresh food 50 to be subjected to quality determination can be handled as long as it is within the range of visible light.

Next, a method for determining the quality of fresh foods will be explained.
As shown in FIGS. 1 and 2, the fresh food quality determination method is a method of determining the quality of the fresh food 50 from image data 51 of the fresh food 50. An image/biochemical index database construction step STEP 1 (S1, STEP is indicated as S in the drawing) for constructing a database (not shown) from the image data 51 and the biochemical index 40 obtained by measuring the fresh food 50 for learning. ), a judgment model learning step STEP 2 in which a judgment model is learned by extracting feature quantities representing color, amount of reflected light, and surface shape using machine learning from the image data 51 as training data in the database, and the target of quality judgment. A fresh food quality determination step STEP3 in which the quality of the fresh food 50 to be subjected to quality determination is determined by estimating the closest determination model from image data 51 for determination obtained by photographing the fresh food 50.

Next, the image/biochemical index database construction step of STEP 1 will be explained.
As shown in FIGS. 2 to 6, in the image/biochemical index database construction step STEP 1, fresh food (yellowtail fillet) 50 that has been landed or harvested for 0 days is prepared, and an illumination 22 of a predetermined color temperature is applied. The image data 51 for learning is created by photographing the image data 51 for learning by applying it to the fresh food for learning 50 with the camera 25. Next, the fresh food for learning 50 is rotated by a predetermined angle in a plan view and the camera 25 Repeat n times to take a picture with the camera 25 and create the learning image data 51, repeat m times to change the color temperature by a predetermined value and take a picture with the camera 25, and create the learning image data 51. Furthermore, the process of photographing the freshly harvested 0-day-old fresh food 50 with the camera 25 every predetermined period of time to create learning image data 51 is repeated p times, and a total of n×m×p training images are captured. An image data creation sub-step STEP1-1 for creating image data 51 is provided.

Furthermore, in the image/biochemical index database construction step STEP 1, biochemical indexes include K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid. These biochemical indicators are measured with a measuring device (biochemical indicator measurement unit 40) to obtain image data for learning in the biochemical indicator measurement sub-step STEP 1-2 and the image data creation sub-step STEP 1-1. 51, and a database construction sub-step STEP 1-3 for constructing a database from the biochemical indexes in the biochemical index measurement sub-step STEP 1-2, and in STEP 1-4, the fresh food 50 for learning has passed a predetermined storage period. If the storage period has not expired (NO), the process ends.

In addition, in detail, in the image/biochemical index database construction step STEP 1, freshly harvested fresh food 50 that is 0 days old is prepared, and a camera is placed in a state where illumination with a predetermined color temperature is applied to the fresh food 50 for learning. 25 to create learning image data 51, and then the learning fresh food 50 is rotated by 360°/32 in planar view and photographed by the camera 25 to create learning image data 51. This process is repeated 32 times, and the color temperature is further divided into 7 levels from 3300K to 5600K, and each color temperature is photographed with the camera 25 to create image data 51 for learning. This process is repeated 14 times to create learning image data 51 by photographing 0-day-old fresh food with the camera 25 every 12 hours, resulting in 7 days worth of learning image data 51. Create. Furthermore, the biochemical index of the fresh food 50 corresponding to each image data 51 is also measured.

Image data 51 of the fresh food 50 photographed in this manner is shown in FIGS. 7 to 12. Figure 7 is an example of an image of a raw tuna fillet on the 0th day of landing.The vertical column divides the color temperature into seven levels from 3300K to 5600K, and the horizontal column shows two fresh foods. 50 is rotated by 360°/32 in plan view. The K value, which is the composition of ATP-related compounds, is 12.8%, and the pH value is 6.16.

FIG. 8 is an example of an image of a raw tuna fillet on the second day of landing. The K value is 16.8% and the pH value is 6.10.

FIG. 9 is an example of an image of a raw tuna fillet on the fourth day of landing. The K value is 24.8% and the pH value is 6.05.

Moreover, FIG. 10 is an example of an image of a frozen tuna fillet on the 0th day of landing. The K value is 40.7% and the pH value is 6.07.

FIG. 11 is an example of an image of a frozen tuna fillet on the second day of landing. The K value is 61.5% and the pH value is 5.98.

FIG. 12 is an example of an image of a frozen tuna fillet on the fourth day of landing. The K value is 83.1% and the pH value is 5.92.

Next, the judgment model learning process of STEP 2 will be explained.
As shown in FIGS. 2, 13, and 14, the judgment model learning step STEP 2 includes an image data input substep STEP 2-1 in which image data 51 of the database is input to the analysis unit 30, and the measured biochemical index is analyzed. In the biochemical index input sub-step STEP 2-2, which is input to the section 30, the analysis section 30 uses machine learning to extract feature quantities expressing the color, amount of reflected light, and surface shape, and extracts the features of the fresh food 50 for learning. A determination model learning sub-step STEP2-3 is provided in which the determination model is learned in correspondence with the elapsed time from the 0th day of fresh harvesting.

Next, the fresh food quality determination process of STEP 3 will be explained.
As shown in FIGS. 2 and 15, the quality determination step STEP 3 of fresh food is a quality determination process in which image data to be determined by photographing the fresh food 50 to be determined by the camera 25 is input to the analysis unit 30. image data input sub-step STEP 3-1, and a judgment model estimation sub-step STEP 3- in which feature quantities are extracted from the quality judgment target image data inputted by the analysis unit 30 and a judgment model having the closest feature quantity is estimated from the database. 2, and a biochemical index output sub-step STEP 3-3 for outputting a biochemical index of the fresh food subject to quality determination using the biochemical index of the determination model estimated in the determination model estimation sub-step STEP 3-2. There is.

Next, learning of the decision model and test results will be explained.
As shown in FIG. 16, when the fresh food 50 was made into yellowtail, the accuracy was 99.7% and the value of the loss function was 0.0018. Note that learning was performed with the number of epochs (number of learning times) set to 100.

As shown in FIG. 17, when the fresh food 50 was squid, the accuracy was 100% and the value of the loss function was 0.001 or less.

As shown in FIG. 18, when the fresh food 50 was tuna, the accuracy was 100% and the value of the loss function was 0.001 or less.

Next, the functions and effects of the fresh food quality determination method and quality determination system described above will be explained.

According to an embodiment of the present invention, an image/biochemical index database is constructed by constructing a database from training image data obtained by photographing fresh foods for learning and biochemical indices obtained by measuring fresh foods for learning. Because it has a process, it is possible to match the biochemical index obtained by measuring fresh food with image data, rather than the intuition of a connoisseur, and determine the quality of fresh food, including freshness, based on the value of the biochemical index. . A judgment model learning process in which a judgment model is learned by extracting feature values representing color, amount of reflected light, and surface shape using machine learning from image data as training data in a database, and a judgment model learning process that uses machine learning to extract features expressing color, amount of reflected light, and surface shape, and learns a judgment model. The process includes a fresh food quality determination process in which the quality of the fresh food subject to quality determination is determined by estimating the closest determination model from the photographed image data for determination. The quality of fresh food can be determined instantly.

Furthermore, the image/biochemical index database construction process involves preparing fresh foods 0 days after landing or harvesting, and photographing the fresh foods with a camera while illuminating them with a predetermined color temperature. We created image data for training, then rotated the fresh food for training by a predetermined angle in a planar view, photographed it with a camera, and created image data for training. This process was repeated n times, and then the color temperature was adjusted. The process of changing only a predetermined value and photographing with a camera to create image data for learning is repeated m times, and then freshly harvested 0-day-old fresh food is photographed with a camera every predetermined period of time to create image data for learning. The image data creation sub-step includes repeating the creation of image data p times to create image data for learning a total of n×m×p times. For this reason, the judgment model is learned from a large amount of training image data consisting of different color temperatures and different angles, so it is not affected by the brightness or darkness of the environment or shooting angle in which the fresh food that is subject to quality judgment is photographed. , quality can be determined accurately. In this way, in a visible light environment, the quality of fresh foods, such as seafood, meat, fruits and vegetables, can be judged using biochemical indicators.

Further, according to an embodiment of the present invention, the biochemical indicators include K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid. In the biochemical index measurement sub-process, these biochemical indexes are measured using measuring instruments, so unlike sensory tests, individual differences do not occur, and no matter who judges the fresh food, the same result can be obtained. This allows for stable and highly accurate quality judgment.

Furthermore, according to the embodiment of the present invention, in the decision model learning sub-step, the analysis unit uses machine learning to extract feature quantities expressing the color, amount of reflected light, and surface shape, and extracts the characteristics of the fresh food for learning. Since the determination model is learned in correspondence with the elapsed time from the 0th day of freshness, the quality of fresh food can also be determined according to the number of days elapsed.

Furthermore, according to the embodiment of the present invention, when the analysis unit extracts features from image data to be subjected to quality judgment inputted by machine learning, a deep learning method using a so-called neural network or an invention invented by the present inventor is used. By using the deep learning method based on "Statistical distance matrix calculation method, statistical distance matrix visualization method, device, and program" disclosed in Japanese Patent Application No. 2022-52171, the learning quality of the decision model is improved. The accuracy of quality determination of fresh foods can be improved.

Furthermore, according to the embodiment of the present invention, the color temperature is divided into seven stages from 3300K to 5600K, and the fresh food for learning is rotated by 360°/32 in plan view for each color temperature. Since the process of photographing with a camera and creating image data for learning is repeated 32 times, the learning quality of the judgment model can be improved and the accuracy of quality judgment of fresh foods can be improved.

In the example, the fresh food 50 was seafood such as yellowtail, squid, and tuna, but it is not limited to this, and the fresh food 50 may be other seafood such as horse mackerel, scallop, and bonito, and furthermore, beef, Meat such as pork and chicken, and fruits and vegetables such as lettuce, cabbage, and apples are also acceptable. That is, the present invention is not limited to the examples as long as the functions and effects of the present invention are achieved.

In addition, in the embodiment, the color temperature of the illumination is divided into 7 stages from 3300K to 5600K, but the color temperature is not limited to this, and the color temperature may be divided into 9 stages from 3000K to 6000K. The temperature range and number of divisions may be varied.

In addition, in the example, the fresh food for learning was rotated in 360°/32 increments in a planar view and photographed with a camera, but it is not limited to this, and it may be taken every 10°, 15°, etc., and the more detailed it is, the more the judgment model The angle at which the camera is rotated may be other values, although the accuracy will be improved. Furthermore, when image data is taken every predetermined time period, the time interval may be not only every 12 hours but also every 6 hours, every 24 hours, etc. Furthermore, the number of days for which data is collected may be changed as appropriate, such as for 4 days, 7 days, or 10 days, depending on the raw material of the fresh food. In addition, in the examples, the biochemical indicators were K value, VBN, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid, but they are not limited to these, and other indicators may be used as long as they are related to quality. .

INDUSTRIAL APPLICATION This invention is suitable for the quality determination technique of fresh food which determines the quality of fresh food from the image data of fresh food.

10...Fresh food quality determination system, 20...Teacher data creation unit, 22...Lighting (light source), 23...Rotary table, 25...Camera, 30...Analysis unit (database, library, artificial intelligence), 40...Biochemical index Measuring section (biochemical index, K value, VBN, pH value, muscle pigment, chlorophyll, carotenoid, ascorbic acid), 50...Fresh foods (yellowtail, squid, tuna, seafood, beef, pork, chicken, meat, lettuce, cabbage, apple, fruits and vegetables), 51...Image data.

[2] Preferably, the image/biochemical index database construction step includes:
The biochemical index includes one or more of the K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid , and these a biochemical index measurement sub-step of measuring the biochemical index using a measuring device;
a database construction sub-step of constructing a database from the image data for learning in the image data creation sub-step and the biochemical index in the biochemical index measurement sub-step;
The process ends when the fresh food for learning exceeds a predetermined storage period.

[3] Preferably, the judgment model learning step includes:
an image data input sub-step of inputting the image data of the database to an analysis section;
a biochemical index input sub-step of inputting the measured biochemical index to the analysis section;
The analysis unit uses machine learning to extract features expressing color, amount of reflected light, and surface shape , and extracts the biochemical index and the elapsed time from the 0th day of freshness of the fresh food for learning. and a decision model learning sub-step of learning a decision model in correspondence with the decision model.

[6] Preferably, a fresh food quality determination system that determines the quality of the fresh food from image data of the fresh food,
Equipped with a teacher data creation section, a biochemical index measurement section, an analysis section, and a quality judgment target shooting display section,
The teacher data creation unit includes a light that illuminates the fresh food with a changeable color temperature, a camera that photographs the fresh food, and a rotating device that is rotatable at a predetermined angle in a plan view and on which the fresh food is placed. Equipped with a stand,
The image data for learning is created by photographing the fresh food for learning with the camera while illumination with a predetermined color temperature is applied to the fresh food for learning, and then the fresh food for learning is viewed from a predetermined angle in plan view. Repeating n times to rotate the image data by the same amount and take a picture with the camera to create the image data for learning,
Further, changing the color temperature by a predetermined value and photographing with the camera to create the image data for learning is repeated m times,
Further, the process of photographing the 0-day-old fresh food with the camera every predetermined time period to create the image data for learning is repeated p times, and the image data for learning is created for a total of n×m×p times. The image data is created;
The biochemical index measuring section includes at least one or more of the following: K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid. It is a measuring device that measures indicators.
The analysis unit includes a database configured to include the image data for learning and the biochemical index corresponding to these image data,
A determination model is learned by extracting feature quantities expressing color, amount of reflected light, and surface shape from the image data as training data of the database using machine learning , and correlating them with the biochemical index .
The quality determination target photographing display unit uses image data for determination obtained by photographing the fresh food to be subjected to quality determination, and the analysis unit estimates the determination model that is closest to the image data for determination. The present invention is characterized in that the quality of the fresh food subject to quality determination is determined and the biochemical index is displayed.

Claims (7)

A fresh food quality determination method for determining the quality of the fresh food from image data of the fresh food, the method comprising:
an image/biochemical index database construction step of constructing a database from learning image data obtained by photographing the fresh food for learning and biochemical indices obtained by measuring the fresh food for learning;
a determination model learning step of learning a determination model by extracting feature quantities representing color, amount of reflected light, and surface shape using machine learning from the image data as training data of the database;
A fresh food quality determination step of determining the quality of the fresh food that is the subject of quality determination by estimating the closest determination model from the image data for determination taken of the fresh food that is the subject of quality determination. have,
The image/biochemical index database construction step includes:
Prepare the fresh food after landing or 0 days after harvesting, and create the image data for learning by photographing with a camera with illumination of a predetermined color temperature applied to the fresh food for learning, Next, the process of rotating the fresh food for learning by a predetermined angle in planar view and photographing it with the camera to create the image data for learning is repeated n times;
Further, changing the color temperature by a predetermined value and photographing with the camera to create the image data for learning is repeated m times,
Further, the process of photographing the 0-day-old fresh food with the camera every predetermined time period to create the image data for learning is repeated p times, and the image data for learning is created for a total of n×m×p times. A method for determining the quality of fresh food, comprising an image data creation sub-step of creating the image data. The method for determining the quality of fresh food according to claim 1, comprising:
The image/biochemical index database construction step includes:
The biochemical indexes include the K value, which is the composition of ATP-related compounds, VBN, which is volatile basic nitrogen, pH value, and muscle pigment, and the biochemical index obtained by measuring these biochemical indexes with a measuring device. An indicator measurement sub-process;
a database construction sub-step of constructing a database from the image data for learning in the image data creation sub-step and the biochemical index in the biochemical index measurement sub-step;
A method for determining the quality of fresh food in which processing is terminated when the fresh food for learning exceeds a predetermined storage period. The method for determining the quality of fresh food according to claim 1 or claim 2,
The judgment model learning step includes:
an image data input sub-step of inputting the image data of the database to an analysis section;
a biochemical index input sub-step of inputting the measured biochemical index to the analysis section;
The analysis unit uses machine learning to extract feature values expressing color, amount of reflected light, and surface shape, and creates a judgment model by matching them with the elapsed time from day 0 of the fresh food for learning. A method for determining the quality of fresh food, comprising: a determination model learning sub-step for learning. The method for determining the quality of fresh food according to claim 3, comprising:
The fresh food quality determination step includes:
a quality determination image data input sub-step of inputting the image data of the quality determination target obtained by photographing the fresh food target of quality determination with a camera to the analysis unit;
a determination model estimation sub-step in which the analysis unit extracts a feature amount from the inputted image data to be determined for quality, and estimates the determination model having the closest feature amount from the database;
and a biochemical index output sub-step of outputting a biochemical index of the fresh food subject to quality determination using the biochemical index of the determination model estimated in the determination model estimation sub-step. Method. The method for determining the quality of fresh food according to claim 1, comprising:
The image/biochemical index database construction step includes:
Prepare the fresh food that has been freshly harvested for 0 days, and photograph the fresh food for learning with a camera while shining a light of a predetermined color temperature on the fresh food for learning to create the image data for learning, and then Repeating 32 times to create the image data for learning by rotating the fresh food for learning by 360°/32 in plan view and photographing it with the camera;
Further, the color temperature is divided into seven stages from 3300K to 5600K, and each color temperature is photographed with the camera to create the image data for learning, and this is repeated seven times,
Furthermore, the above-mentioned fresh food that is 0 days old is photographed with the above-mentioned camera every 12 hours to create the above-mentioned image data for learning, which is repeated 14 times to create the above-mentioned image data for 7 days worth of learning. A method for determining the quality of fresh food. A fresh food quality determination system that determines the quality of the fresh food from image data of the fresh food, the system comprising:
Equipped with a teacher data creation section, a biochemical index measurement section, an analysis section, and a quality judgment target shooting display section,
The teacher data creation unit includes a light that illuminates the fresh food with a changeable color temperature, a camera that photographs the fresh food, and a rotating device that is rotatable at a predetermined angle in a plan view and on which the fresh food is placed. Equipped with a stand,
The image data for learning is created by photographing the fresh food for learning with the camera while illumination with a predetermined color temperature is applied to the fresh food for learning, and then the fresh food for learning is viewed from a predetermined angle in plan view. Repeating n times to rotate the image data by the same amount and take a picture with the camera to create the image data for learning,
Further, changing the color temperature by a predetermined value and photographing with the camera to create the image data for learning is repeated m times,
Further, the process of photographing the 0-day-old fresh food with the camera every predetermined time period to create the image data for learning is repeated p times, and the image data for learning is created for a total of n×m×p times. The image data is created;
The biochemical index measurement unit is a measuring device that measures biochemical indexes including at least K value, which is the composition of ATP-related compounds, VBN, which is volatile base nitrogen, pH value, muscle pigment, chlorophyll, carotenoid, and ascorbic acid. and
The analysis unit includes a database configured to include the image data for learning and the biochemical index corresponding to these image data,
A determination model is learned by extracting feature quantities expressing color, amount of reflected light, and surface shape using machine learning from the image data as training data of the database,
The quality determination target photographing display unit uses image data for determination obtained by photographing the fresh food to be subjected to quality determination, and the analysis unit estimates the determination model that is closest to the image data for determination. A system for determining the quality of fresh foods, characterized in that the system determines the quality of the fresh foods that are subject to quality determination, and displays the biochemical indicators. 7. The fresh food quality determination system according to claim 6,
The illumination can be irradiated with the color temperature changed in seven stages from 3300K to 5600K,
The rotary table is a fresh food quality determination system that can rotate by 360°/32 in a plan view.

Images