KR102380289B1 - Apparatus and method for determining beef grade - Google Patents

Abstract

In the beef grading method performed by the beef grading device according to an embodiment of the present invention, the method comprising: receiving a beef photographed image; pre-processing the received beef image; And analyzing the pre-processed beef image using a machine learning model to determine the grade of beef, the pre-processing step is to determine the fat and lean meat of beef using spectroscopy in the VNIR (Visible and Near-Infrared) region. It is intended to provide a beef grading method, including the step of setting a region of interest (RoI).

Description

Beef grading device and method {APPARATUS AND METHOD FOR DETERMINING BEEF GRADE}

Beef grading apparatus and method according to the present invention relates to a beef grading apparatus and method for determining the grade of beef using spectroscopy and machine learning.

In general, beef is classified immediately after slaughter and sold at different prices depending on the grade. The grade of beef is judged through meat quality and quantity grade, and the method of judgment is based on the naked eye of a professional grader.

However, since it is difficult to accumulate quantified data for each grading item for grading, there is a problem in that it is not possible to secure the objectivity of the grading result. In addition, there are disadvantages in that it is difficult to train a professional grade judge because the time required for grade determination is very long and experience is important.

In order to solve this problem, research on a technology for automatically classifying grades through image analysis is continuing, and interest is increasing with the development of digital image technology. However, since the cross section of beef is mixed with lean meat and fat, and the separation between them is not clear, the boundary line of the judgment area extracted by conventional techniques is significantly different from the boundary line extracted by a professional grader.

Therefore, it is necessary to overcome these differences and develop a device that automatically determines the same grade as the beef grade determined by a professional grader.

Republic of Korea Patent Publication No. 10-2013-0017144 (Title of the invention: Beef history information providing system and method using Qr code)

Beef grade determination apparatus and method according to the present invention, beef graders, researchers, and general consumers using a camera attached to a smart phone to check the distribution of beef back fat (marbling), and grade (1++, 1+ , 1st, 2nd & 3rd grade) aims to obtain objective judgment values that can be helpful in judgment.

In the beef grading method performed by the beef grading device according to an embodiment of the present invention, the method comprising: receiving a beef photographed image; pre-processing the received beef image; And analyzing the pre-processed beef image using a machine learning model to determine the grade of beef, the pre-processing step is to determine the fat and lean meat of beef using spectroscopy in the VNIR (Visible and Near-Infrared) region. It is intended to provide a beef grading method, including the step of setting a region of interest (RoI).

In this embodiment, the pre-processing step may provide a beef grading method, comprising the steps of shaking the beef image, correcting distortion, color analysis, and histogram equalization.

In this embodiment, the machine learning model may provide a beef rating determination method, which is a model that has been previously trained on a beef image to which the rating is mapped.

In this embodiment, receiving a predicted grade of beef; transmitting the graded image, the judgment grade, and the predicted grade to the beef grading server; And it is possible to provide a beef back fat grading method comprising the step of receiving the accuracy of the beef grading using a machine learning model determined by comparing the judgment grade and the predicted grade from the grading server.

In this embodiment, it is possible to provide a beef back fat grading method, including the step of learning a machine learning model based on the mapping of the graded image with the prediction grade.

According to an embodiment of the present invention, in the beef grading device, the beef grading program is recorded memory; and a processor executing a program recorded in the memory, wherein the processor receives, according to the execution of the program, a beef shot image captured by a camera, pre-processes the received beef image, and converts the pre-processed beef image into a machine learning model To determine the grade of beef by analyzing using, the pre-processing includes determining the fat and lean meat of beef using spectroscopy in the VNIR (Visible and Near-Infrared) area and setting the RoI (Region of Interest) Beef grade It is intended to provide a judgment device.

In this embodiment, the pre-processing may provide a beef grading device, including shaking, distortion correction, color analysis, and histogram equalization of the beef image.

In this embodiment, the machine learning model may provide a beef grade determination device, which is a model that has previously learned a grade-mapped beef image.

In this embodiment, receiving the predicted grade of beef, sending the graded image, the judgment grade and the predicted grade to the beef grading server, and machine learning determined by comparing the judgment grade and the predicted grade in the grading server A beef grading device may be provided, comprising receiving an accuracy of beef grading using the model.

In this embodiment, it is possible to provide a beef back fat grading device, comprising learning a machine learning model based on mapping the graded image with the prediction grade.

Beef grade determination apparatus and method according to the present invention, beef graders, researchers, and general consumers using a camera attached to a smart phone to check the distribution of beef back fat (marbling), and grade (1++, 1+ , 1, 2 & 3) objective judgment values that can be helpful in judgment can be obtained.

1 is a configuration diagram showing the configuration of a beef grading device according to an embodiment of the present invention.
Figure 2 is a flow chart showing the sequence of the beef grading method according to an embodiment of the present invention.
3 is a view showing the setting of RoIs of muscle parts and fat parts by grade in beef images according to an embodiment of the present invention.
Figure 4 is a view showing a spectral spectrum for each beef grade, each part of the beef image according to an embodiment of the present invention.
5 is a diagram showing a result of analysis of a main component of fat in a near-infrared (NIR) band of a beef image according to an embodiment of the present invention.
6 is a view showing the determination result of fat and lean meat for each NIR band grade of the beef image according to an embodiment of the present invention.
7 is a view showing the result of the mixed principal component analysis of fat and lean meat in the VNIRS band of the beef image according to an embodiment of the present invention.
8 is a view showing the results of analysis of the main component of fat and lean meat in the VNIRS band of the beef image according to an embodiment of the present invention.
9 is a diagram showing the structure of a machine learning model for determining beef grade according to an embodiment of the present invention.
10 is a diagram showing accuracy and error as a result of applying a machine learning model to beef rating determination according to an embodiment of the present invention.
11 is a diagram showing a user interface of a beef rating determination method program according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, an embodiment of the present invention will be described in detail so that a person skilled in the art to which the present invention pertains can easily implement it. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And, in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that other components may be further included rather than excluding other components unless otherwise stated.

Hereinafter, it will be described with respect to the beef grading device according to an embodiment of the present invention.

The beef rating determination device 100 may be a device for photographing beef using a camera and a flash, and analyzing the photographed image through machine learning to determine a grade.

On the other hand, the beef rating determination device 100 may be a terminal such as a smart phone, tablet. In the present invention, a 'terminal' may be a wireless communication device with guaranteed portability and mobility, for example, any type of handheld-based wireless communication device such as a smart phone, tablet PC, or notebook computer.

In addition, the 'terminal' may be a wired communication device such as a PC that can connect to another terminal or server through a network.

In addition, the network refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW). : World Wide Web), wired and wireless data networks, telephone networks, and wired and wireless television networks.

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

1 is a configuration diagram showing the configuration of a beef grading device according to an embodiment of the present invention.

Meanwhile, the beef rating determination apparatus 100 may include a communication module 110 , a memory 120 , a processor 130 , and a database (DB: 140 ).

The communication module 110 provides a communication interface to the beef rating determination device 100 in conjunction with a communication network, and may serve to transmit and receive data to and from the beef rating determination server. Here, the communication module 110 may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals through wired/wireless connection with other network devices.

Memory 120 may be a beef grading method program is recorded. In addition, the memory 120 may perform a function of temporarily or permanently storing data processed by the processor 130 . Here, the memory 120 may include a volatile storage medium or a non-volatile storage medium, but the scope of the present invention is not limited thereto.

The processor 130 may control the entire process performed by the beef grading method program in the beef grading device 100 . Each step of the process performed by the processor 130 will be described later with reference to FIGS. 2 to 11 .

Here, the processor 130 may include all kinds of devices capable of processing data, such as a processor. Here, the 'processor' may refer to, for example, a data processing device embedded in hardware having a physically structured circuit to perform a function expressed as a code or command included in a program. As an example of the data processing apparatus embedded in the hardware as described above, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit) and a processing device such as a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto.

The database 140 may store data required to execute a beef rating determination method program, including beef image data learned by machine learning to be described later.

Hereinafter, a method for determining a beef rating according to an embodiment of the present invention will be described.

In the beef rating determination method, the step (S210) of receiving a beef photographed image may be performed first. The beef photographing image may be obtained by photographing beef using a camera and a flash provided in the beef rating determination device 100 .

Next, receiving the predicted grade of beef (S220) may be performed. For example, the grader may input the predicted grade obtained by visually observing the beef and determining the grade in the beef grade determination apparatus 100 .

Next, a step (S230) of pre-processing the received beef image may be performed.

The pre-processing step may include stabilizing, distortion-correcting, color analysis and histogram equalization of the beef image.

Here, one of the distortions occurring in the image, radial distortion, means that the position of the object in the image is different from the position of the object in the real space. Since the present invention deals with the fine intramuscular fat of beef, it is necessary to accurately position even small parts of the image. Therefore, it is necessary to correct the distortion of the sample image. Radial distortion occurs in the radial direction due to the refraction of light passing through the lens and defects in the lens, and it appears as negative (Barrel) distortion and positive (Pincushion) distortion depending on the type of distortion that occurs. The distortion ratio of the image was calculated through [Equation 1] and [Equation 2] below.

[Equation 1]

[Equation 2]

In [Equation 1], X _D , Y _D are coordinate values in the distorted image, and X _t , Y _t are coordinate values in the corrected image. If a point in the distorted image is P(X _D ,Y _D ) and a point in the image whose distortion is corrected is P'(X _t ,Y _t ), the difference in the X-axis coordinates ( The difference between ΔX) and the Y-axis coordinates (ΔY) and the linear distance (ΔR) between the two points (P, P′) can be calculated. Since the position of the ideal point is the position of P', the distortion rate is calculated by [Equation 2].

In addition, the pretreatment step may include a step of determining the fat and lean meat of beef using spectroscopy in a visible and near-infrared (VNIR) region and setting a region of interest (RoI).

Here, the spectroscopy method is a method of experimentally obtaining physical and chemical information of an analysis target by measuring the intensity of light absorption and emission as a function of light energy. A single atom/molecule or component within a substance interacts with light by absorbing light, scattering light of a specific wavelength, or emitting fluorescence. It is possible to study various properties of atoms/molecules and materials by observing the phenomenon of interaction with light.

3 is a view showing the setting of RoIs of muscle parts and fat parts for each grade of beef image according to an embodiment of the present invention.

Referring to FIG. 3 , in order to investigate the quality characteristics using hyperspectral images, an experiment was performed to discriminate fat and lean meat using spectral information of visible and near-infrared wavelengths.

The wavelength bands in the experiment were visible and near infrared (400 - 1,700 nm).

Figure 4 is a view showing the spectral spectrum for each beef grade, each part of the beef image according to an embodiment of the present invention.

Referring to FIG. 4 , although the spectral shape of the fat component and the lean meat component is similar, it can be confirmed that there is a difference in reflectivity, and in particular, it can be seen that there is a difference in the spectral shape at 1450 nm, which is known as an overtone band related to protein.

5 is a diagram showing a result of analysis of a main component of fat in a near-infrared (NIR) band of a beef image according to an embodiment of the present invention. 6 is a view showing the determination result of fat and lean meat for each NIR band grade of the beef image according to an embodiment of the present invention.

Referring to FIGS. 5 and 6 , it can be seen that the discriminant results for each region are graphically represented by using a linear discriminant analysis (LDA) method. There is a difference in accuracy for each grade. In particular, in the case of grade 2, it can be seen that the discrimination accuracy is lowered to an accuracy of 0.77. It has a lot of muscle components, so the area of fat is smaller than other parts, and the depth is shallow.

7 is a view showing the result of the mixed principal component analysis of fat and lean meat in the VNIRS band of the beef image according to an embodiment of the present invention. 8 is a view showing the results of analysis of the main component of fat and lean meat in the VNIRS band of the beef image according to an embodiment of the present invention.

7 and 8, in the visible and near-infrared region (400-1000 nm), it was found that even in the case of grade 2, the discrimination accuracy was improved to 0.83, and in grades 1++ to 1, it was 0.91 to 0.95, high discrimination performance. was shown. Therefore, it was found that the visible-near-infrared region was more effective than the near-infrared region in order to discriminate the beef back fat.

9 is a diagram showing the structure of a machine learning model for determining beef grade according to an embodiment of the present invention. 10 is a diagram showing accuracy and error as a result of applying a machine learning model to beef rating determination according to an embodiment of the present invention.

Referring to FIG. 9 , a machine learning model that learns and discriminates using a sample made by cutting a color image to a size of 161 × 161 pixels can be confirmed.

Next, a step (S240) of determining the grade of beef by analyzing the preprocessed beef image using a machine learning model may be performed.

Here, the machine learning model may be a model in which a grade-mapped beef image is trained in advance. A machine learning model can be composed of multiple CNN layers. CNN is a method of an artificial neural network in which a feature of an input image is generated using various filters that perform a convolution operation, and learning is performed based on this. Depending on the size and stride of the filter, the features of the input image are generated differently, and different learning results can be derived according to the layer configuration for each step. In the present invention, the size of the filter in all convolution layers is set to 3 x 3, and the stride is set to 1. ReLU (Rectified linear unit), Batch normalization, and Dropout were used as the main algorithms in CNN construction. After the convolution operation, ReLU was used as the activation function of the layer. The use of ReLU can train faster and with higher performance than other activation functions. Batch size refers to the number of input data to be learned at one time. That is, even within one learning, the learning process is changed according to the setting of the batch size. Batch normalization normalizes the results of each batch as input data, and uses their averages in the experimental process. Batch normalization reduces covariate shift and enables the design of deeper layers. A dropout may be involved in the activation of a neuron in a corresponding layer to control signal transmission to the next layer. Through this, overfitting can be prevented and high performance results can be obtained.

Referring to FIG. 9 , the machine learning model according to an embodiment of the present invention has four convolutional layers and two fully-connected layers, and 2 for each convolutional layer. Maxpooling of x 2 size was used as subsampling. ReLU is used as the activation function, and batch normalization is applied to all convolutional layers.

A fully connected layer is connected to all neurons in the layers immediately preceding it. In the first fully connected layer, 4,096 neurons in the previous layer are connected to 1,000. The last fully connected layer is connected to 4 categories, which is the number of final outputs after dropout is applied at a rate of 0.5, and at this time, it was configured to use Softmax as an activation function.

Referring to Figure 10, as a result of learning for 60 times of beef rating determination of the machine learning model according to an embodiment of the present invention, the final accuracy was 0.94, and the error was 0.14.

Next, a step (S250) of transmitting the graded image, the judgment grade, and the prediction grade to the beef grade determination server may be performed. Next, the step (S260) of receiving the accuracy of the beef grading using the machine learning model determined by comparing the judging grade and the prediction grade from the grading server may be performed. Next, the step (S270) of learning the machine learning model based on the mapping of the graded image with the prediction grade may be performed. In this way, the accuracy of the machine learning model can be continuously improved.

11 is a diagram showing a user interface of a beef rating determination method program according to an embodiment of the present invention.

Referring to FIG. 11 , pre-processing is performed to extract only the region to be determined from the image taken by the beef grading device 100, and deep learning pre-learned on an image cut out of the pre-processed image in a mosaic shape to 161 × 161 pixels The model can be used to determine the grade. The judgment result indicates the probability for each grade. In addition, if you want to correct the result of shooting and determining, if you transmit the prediction grade to the beef rating server, the accuracy can be improved by reflecting it in the modification of the machine learning model.

On the other hand, the beef rating determination method according to an embodiment of the present invention may be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The above description is merely illustrative of the technical idea of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: beef grading device
110: communication module
120: memory
130: processor
140: database

Claims (10)

In the beef grading method performed by the beef grading device,
Receiving a beef photographed image;
Pre-processing the received beef image by shaking correction, distortion correction, color analysis and histogram equalization; and
Comprising the step of analyzing the pre-processed beef image using a machine learning model that is a pre-trained model of the grade-mapped beef image to determine the grade of the beef,
The pre-processing step is
In the VNIR (Visible and Near-Infrared) region, using a spectroscopy method using hyperspectral images, the fat and lean meat of beef are identified, and the RoI (Region of Interest) is set to confirm the distribution of beef backfat, and a mosaic form with a predetermined pixel size comprising the step of cutting with
The step of determining the grade of the beef,
transmitting the graded image, judgment grade and prediction grade to a beef grade determination server; and
Beef rating method further comprising the step of receiving a prediction rating comprising the step of receiving the accuracy of the beef rating using the machine learning model determined by comparing the rating and the prediction rating from the rating server.
delete delete delete The method of claim 1,
and training the machine learning model based on mapping the graded image with the prediction grade.
In the beef grading device,
a memory in which a beef grading program is recorded; and
a processor executing the program recorded in the memory;
The processor according to the execution of the program,
Receiving a beef shot image taken with a camera, pre-processing the received beef image by shaking correction, distortion correction, color analysis and histogram equalization, and pre-learning the pre-processed beef image with a grade mapped beef image It is a model Determine the grade of the beef by analyzing it using a machine learning model,
The pre-processing is to determine the distribution of beef back fat by determining the fat and lean meat of beef using a spectroscopy method using a hyperspectral image in the VNIR (Visible and Near-Infrared) region, setting the RoI (Region of Interest), and checking the predetermined pixel including cutting to size in the form of a mosaic;
The machine that receives the predicted grade of beef, sends the graded image, the judgment grade, and the predicted grade to a beef grading server, and is judged by comparing the judgment grade with the predicted grade in the grading server Beef grading device, characterized in that receiving the accuracy of beef grading using the learning model.
delete delete delete 7. The method of claim 6,
Including training the machine learning model based on the mapping of the graded image with the prediction grade, beef grade determination device.

Images