JP2006239602A - Grading method for fruit and vegetable - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a grading method for fruits and vegetables which is substituted for determination by human eyes. <P>SOLUTION: The grading method for fruits and vegetables for determining their grades by their images obtained by photographing fruits and vegetables has the process of making a two-dimensional color characterization maps peculiar to the kinds of fruit and vegetables by self-organization by an interconnecting neural network using data converted to the three primary colors of red (R), green (G) and blue (B) obtained from the respective elements of the images photographing fruits and vegetables to be subjected to the grade determination, and an HSV system, and the process of outputting the grades of the desired fruit and vegetables by deciding a plurality of amounts of two-dimensional characterization by samples obtained by sorting the fruits and vegetables to the grades from the color/position information of the object materials which are obtained using the two-dimensional characterization maps and labeling, and by acting an N layer neural network to the amounts of two-dimensional characterization for the grades. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method for classifying fruits and vegetables such as apples and tomatoes.

Fruits and vegetables such as apples and tomatoes are classified according to the color unevenness, scratches, shape, etc., and sorted according to each grade before shipment.
For example, in the case of fruits, for example, in the case of fruits, the color level is determined by comparing the total amount continuously measured for the amount of color (one-dimensional color amount) contained in a small amount of a circular slice of interest by a color sensor and a set threshold value. It was the method of judging.
The method using a color sensor is not a problem when the distribution of colors is universal, but when there is locally distributed color unevenness, this one-dimensional method cannot distinguish the color unevenness. However, when such a two-dimensional judgment is required, it is necessary to rely on human judgment, and much time and effort are required for class discrimination.

There have been various proposals for determination methods using image processing. For example, Japanese Patent Application Laid-Open No. 2000-242791 uses a frequency having a specific spectral reflectance in order to reduce the reflection portion of illumination. A method is described.
Japanese Patent Application Laid-Open No. 9-29185 describes a method for determining the quality of a peel net pattern such as melon.
Furthermore, Japanese Patent Laid-Open No. 5-2632 describes a method for determining a grade using color information obtained from an imaging camera using a neural network technique.
However, these conventional techniques have not reached a level that can be substituted for judgment by human eyes, and have not been put into practical use.
JP 2000-242791 A JP-A-9-29185 Japanese Patent Laid-Open No. 5-2632

  An object of the present invention is to solve the above-mentioned problems of the prior art and to provide a fruit and vegetable grade determination method that can be substituted for determination by human eyes.

As a result of intensive studies in order to solve the above-described problems, the present invention uses a two-dimensional feature value from the color / position information of a target object obtained by a two-dimensional color feature map and a labeling process. The present invention provides a method for discriminating the grade of fruits and vegetables that can be substituted for judgment, and the gist thereof is as follows.
(1) A method for determining the grade of fruit and vegetables that discriminates the grade based on an image of the fruit and vegetables.
Expressing each pixel of an image obtained by imaging fruits and vegetables to be graded with the three primary colors of red (R), green (G), and blue (B), or converting the color of each pixel to the HSV system;
Using the data converted to the primary colors of red (R), green (G), and blue (B) obtained from each of the pixels, or the HSV system, self-organization is performed by an interconnection neural network, and the kind of Creating a two-dimensional color feature map specific to fruits and vegetables;
A plurality of two-dimensional feature quantities are determined by using the sample classified fruit and vegetable by grade using the color / position information of the target object obtained by using the two-dimensional color feature map and labeling process, Configuring the N-layer neural network so that each grade is determined when the N-layer neural network is applied to the dimension feature,
And a step of outputting the grade of the fruit and vegetables by causing the neural network to act on a two-dimensional color feature obtained by imaging the arbitrary fruit and vegetables. Method.
(2) The result obtained by the two-dimensional color feature map and the labeling process includes shape information, and is applied to shape or scratch selection in fruit and vegetables, and the grade of fruit and vegetables according to (1) How to determine.

ADVANTAGE OF THE INVENTION According to this invention, the grade discrimination method of the fruit and vegetables which can substitute for judgment by human eyes by using a two-dimensional feature-value can be provided.
Specifically, the present invention enables two-dimensional analysis of an image by a completely new series of intelligent image processing methods that have not been conventionally performed. From various two-dimensional feature values obtained based on the two-dimensional analysis, human judgment has been made so far. It is possible to automate the classification of fruits and vegetables that relied on the food.

The best mode for carrying out the invention will be described in detail with reference to FIGS.
FIG. 1 is a diagram exemplifying a processing flow of a method for discriminating fruits and vegetables according to the present invention.
First, each pixel of an image obtained by picking up fruits and vegetables to be graded is expressed by the three primary colors of red (R), green (G), and blue (B), or the color of each pixel is converted to the HSV system ( S-1).
One kind of fruit to be sorted by grade may be captured indiscriminately enough to create a two-dimensional color feature map, and each pixel may be displayed with three primary colors. By expressing colors with three elements of H (Hue: Hue), S (Saturation: Saturation), and V (Value: Lightness), the pixel color is closer to human perception than the RGB color space. In addition, a highlight portion in which light is totally reflected and appears white in the image can be identified by hues H and S.

Next, self-organization is performed by an interconnected neural network using data converted into the three primary colors of red (R), green (G), and blue (B) obtained from each pixel, or the HSV method. A two-dimensional color feature map peculiar to the kind of fruit and vegetable is created (S-2).
Here, self-organization is also referred to as “unsupervised learning” and means that the internal structure is adapted to the external world even though there is no signal (teacher signal) related to the internal structure from the external world.
The interconnected neural network is a model in which the input layer of the feature map and the competitive layer are completely connected, and each input unit is connected to all the units of the competitive layer. All connections have a connection weight, and self-organization is performed by updating this according to the input vector.
In addition, a plurality of two-dimensional features are determined from the color / position information of the target object obtained by using the two-dimensional color feature map and a labeling process based on a sample obtained by classifying the fruits and vegetables according to the grade, The N-layer neural network is configured so that each grade is determined when the N-layer neural network is applied to the feature amount (S-3).
By performing clustering to recognize and classify images of fruits and vegetables by grade using a two-dimensional feature map, and then labeling the connected pixels together and assigning different labels to each area A plurality of two-dimensional feature values are determined from the obtained color / position information of the target object.
Here, the plurality of two-dimensional feature values used in the present invention means a plurality of feature values obtained from a two-dimensional image. In the present invention, the plurality of two-dimensional feature values are conventionally used by using the plurality of two-dimensional feature values. It is possible to automatically determine locally distributed color unevenness that could not be determined by one-dimensional information such as the color amount.

Then, the neural network is applied to the two-dimensional color feature obtained by imaging any of the fruits and vegetables and outputs the grade of the fruits and vegetables (S-4).
Furthermore, the results obtained by the two-dimensional color feature map and the labeling process include shape information, and by applying it to the shape or flaw selection in fruit and vegetable vegetables, it is possible to automatically perform grade discrimination based on shapes other than color unevenness. Can do.
Note that automatic discrimination in the machine field can be realized by applying the present invention to the shape discrimination of machine parts and machine products.

FIG. 2 is a diagram exemplifying an embodiment of the method for determining the grade of fruit and vegetable vegetables according to the present invention, and is a diagram showing a procedure of a class determination process for an unidentified image at the time of completion.
These show a flow of a series of automatic processes on a computer after an image obtained by picking fruits and vegetables is taken into the computer as two-dimensional image data.
The first step is an RGB display target image for top-surface shooting. The second step is conversion from an RGB image to an HSV color space image. As described above, the HSV image is an expression method based on human color perception, and this expression enables color correction suitable for human perception. There is a one-to-one correspondence between HSV and RGB.

ここで、iとjはそれぞれ競合層と入力層のユニット番号である。
勝者ユニットとその近傍の結合定数の更新の仕方は、学習率をαとして次の式で行われる。

また、その学習率αと、勝者ユニット近傍幅ｄは次のように更新する。

ここで、ｔは学習回数の変数であり、Ｔは全学習回数である。 The third step is a color clustering determination process. An interconnected neural network (NN) self-organizing feature map clusters inputs from the HSV color space for each target color. The procedure is illustrated below.
The self-organizing map searches for the winning unit in the competitive layer that minimizes the distance (A) between the input vector E of the input layer unit and the load vector Wi (t) of the competitive layer unit. This is a method of organizing while updating W.

Here, i and j are unit numbers of the competitive layer and the input layer, respectively.
The method for updating the winning unit and the coupling constant in the vicinity thereof is performed by the following formula, where the learning rate is α.

Further, the learning rate α and the winner unit neighborhood width d are updated as follows.

Here, t is a variable of the number of learning times, and T is the total number of learning times.

The labeling in the fourth step is a process of automatically confirming the separation of the color area, and is a process of grouping connected pixels together and assigning a different label to each area. Here, the control is performed for each color in consideration of eight connections surrounding the target pixel. The labeling algorithm is different from the background color: 1) all of the four previously scanned pixels at the top (3 pixels) and left (1 pixel) of the target pixel are different from the target pixel; and 2) the four scanned pixels If any of them is the same as the target pixel, then 3) If they are the same, labeling is performed while correcting the labels of the pixels of the same color. FIG. 3 shows an example of images before and after labeling.
In the fifth step, a plurality of two-dimensional feature quantities effective for discrimination are obtained from the confirmed two-dimensional information of the position and color of the object.
For example, as a feature amount for discriminating color unevenness of apples, for example, F1. Red area size, F2. Average value of saturation in the red region, F3. The size of the uneven color area, F4. Number of uneven color areas, F5. Circularity, F6 to F9. It is preferable to set the degree of scattering of the uneven color region (4 steps depending on the size).

The sixth step is a determination process for determining a grade from a plurality of two-dimensional feature values.
As a discriminating means in the present invention, an N-layer neural network (NN) is used, but it is a learning machine that imitates the brain, and is composed of several layers in which several units are grouped, and the units between each layer are connected. It is preferable to use a hierarchical perceptron that performs learning by updating the bond weight of each bond.
For example, when discriminating the color irregularity of an apple, the feature values of the aforementioned F1 to F9 are obtained, the value is used as an input vector of a three-layer neural network (NN), and learning is performed using a teacher signal as a corresponding grade. .
As a learning method, for example, by using the BP (Back Propagation) method that updates the connection weight by the steepest descent method from the error between the output and the teacher signal, the input vector is encoded and linearly separated. Possible input patterns can be learned.
The number of units is, for example, 9 input layers (number of features), 14 hidden layers, 5 output layers (number of classes), and the learning end condition is ε = 1.0 × 10 ⁻⁵ .
Among these processes, the third, fourth, and fifth steps each independently create a learning device and feature variables suitable for each purpose.
If a two-dimensional image with an unknown grade is taken into the above-described series of discrimination processing systems after learning, the grade discrimination can be performed automatically.

FIG. 4 shows the result of the grade discrimination for the uneven color of the apple using the method of the present invention.
In FIG. 4, the upper row shows apples with special grades and the lower row shows unmarked apples. The left end is the original image, the middle is after clustering, and the right end is the image after labeling.
As a result of class discrimination for 25 images using the grade discrimination method of the present invention, all the correct discrimination can be performed as in the case of a skilled human discrimination result, and the effect of the present invention was confirmed.

It is a figure which illustrates the processing flow of the grade discriminating method of the fruit and vegetables in this invention. It is a figure which illustrates embodiment of the grade discriminating method of the fruit and vegetables in this invention. It is a figure explaining the labeling used for the grade discrimination | determination method of the fruit and vegetables of this invention. It is a figure which shows the Example which applied the grade discrimination | determination method of the fruit and vegetables of this invention to the apple.

Claims (2)

A method for determining the grade of fruit and vegetables that discriminates the grade from an image obtained by imaging the fruit and vegetables,
Expressing each pixel of an image obtained by imaging fruits and vegetables to be graded with the three primary colors of red (R), green (G), and blue (B), or converting the color of each pixel to the HSV system;
Using the data converted to the primary colors of red (R), green (G), and blue (B) obtained from each of the pixels, or the HSV system, self-organization is performed by an interconnection neural network, and the kind of Creating a two-dimensional color feature map specific to fruits and vegetables;
A plurality of two-dimensional feature values are determined using samples classified by grade of the fruits and vegetables using the two-dimensional color feature map and color / position information of a target object obtained by using a labeling process, Configuring the N-layer neural network so that each grade is determined when the N-layer neural network is applied to the two-dimensional feature amount; and
A step of outputting the grade of the fruits and vegetables by causing the neural network to act on the plurality of two-dimensional color features obtained by imaging the fruits and vegetables. Grade discrimination method.   2. The method according to claim 1, wherein the result obtained by the two-dimensional color feature map and the labeling process includes shape information, and is applied to discrimination of a shape or a flaw in fruit vegetables. .

Images