JP5888639B2 - Color image capture and analysis system - Google Patents

Description

The present invention relates to a color image capturing / analyzing system. Specifically, the present invention relates to a color image imaging system and / or a color image analysis system. More specifically, the present invention relates to a color image imaging system and / or a color image analysis system based on Color Histogram and CDE for removing or reducing surface reflection of an object.

The quality control and evaluation of biological minerals such as pearls and the cultivation / quality control and evaluation of agricultural products often depend on the experience and intuition of skilled workers, and it is difficult to quantify clear judgment factors and standards.
In particular, the appearance of the color, shape, size, etc. of the agricultural products is an important quality factor that determines the commercial value of the agricultural products together with the taste and texture. In addition, with the declining birthrate and diversification of foods, the distribution and sale of cut products is thriving, and the introspection of agricultural products is also regarded as important. As described above, time-series monitoring of the growth process, such as cultivation / quality control of agricultural products, evaluation of appearance / introspection, etc., is performed by visual observation of skilled workers in the field, and depends on judgment factors and judgment criteria based on experience and intuition. Therefore, objectivity and reproducibility are issues.
Furthermore, recently, the industrialization of agriculture has attracted attention, and the introduction of plant factories and the like has been studied, but environmental control is the center. When agricultural products are regarded as products, the construction of physical characteristics (specs) desired by agricultural products processors and consumers, the establishment of quantitative evaluation including their quantification, line design, and the like are issues.

Therefore, management and evaluation using optical devices have been attempted, but management and evaluation by human eyes is not a process on computer software by algorithms such as partial summation and image processing. This is often done with the overall impression. Therefore, instead of partial evaluation using a spectrocolorimeter or the like, it is desired to establish management and evaluation close to human eyes by image analysis (color analysis and shape analysis).

In order to analyze the surface color of the object, it is important to faithfully capture the surface image of the object. For this reason, the camera system and the illumination system are extremely important, and it is important to shield the image capturing unit from outside light. In particular, removing or reducing surface reflections is a challenge, and therefore diffuse light sources have been used. As an imaging method using a diffused light source, there is known a method in which a direct circular diffused light source is attached to a camera for photographing, and some of the inventors have used light from a standard fluorescent lamp, which is originally a diffused light source. Then, a method of further increasing the degree of diffused light using a diffuse reflector was constructed (Non-Patent Document 1).

However, since biological minerals such as pearls have a remarkable surface gloss, and agricultural products have a remarkable surface color unevenness and irregularities in addition to the surface gloss, the above-mentioned conventional technique has eliminated or reduced the surface reflection of the object. Color image measurement was difficult.

Among the appearance and introspection qualities of agricultural products, color is an important index that determines quality, especially because there are qualitative variations in hue, quantitative variations in saturation, spatial distribution of colors, etc. Analytical methods have been developed. Some of the inventors have shown the possibility of variety identification by color image analysis on the strawberry surface and cross section using the HSL color space (Non-patent Document 2). In Non-Patent Document 3, strawberry color analysis is performed based on the value of a * in the L * a * b * color system, and combined with the shape and size analysis results, strawberry grade determination is performed. A method of performing is disclosed.

However, in Non-Patent Documents 2 and 3, only the overall value and appearance frequency of the color are targeted for analysis, and there is a problem that the spatial distribution information of the fruit color on the surface and cross section of the strawberry is discarded. .

Motonaga Yoshitaka, Kameoka Koji, Hashimoto Atsushi, Journal of Agricultural Machinery Society, 59 (4), 21-28 (1997). H. Kitamura, T. Mori, A. Hashimoto, and T. Kameoka, Distinction of strawberry cultivars based on fruit characters evaluated by image analysis, Jido Seigyo Rengo Koenkai (CD-ROM), 48, L2-15 (2005). X. Liming, and Z. Yanchao, Automated strawberry grading systembased on image processing, Computers and Electronics in Agriculture, 71, S32 – S39 (2010). J. Sun, X. Zhanng, J. Cui, L. Zhou, Pattern Recognition Letters, 27, 1122-1126 (2006). C. E. Shannon, A mathematicaltheory of communication, Bell Syst Tech (1948). A. Hashimoto, H. Kondou, Y. Motonaga, H. Kitamura, K. Nakanishi, T. Kameoka, 1st World Congress of Computers in Agriculture and Natural Resources, Zazueta, F. and Xin, J. ed., 70-77 ( 2001).

The present invention aims at quality control and evaluation of biological minerals and cultivation / quality control and evaluation of agricultural products that are close to human eyes, and provides a color image imaging / analysis system. In particular, a color image imaging system that removes or reduces surface reflections of an object is provided. A color image analysis system based on Color Histogram and CDE is also provided.
The present invention provides a quality control / evaluation apparatus for biominerals using the color image capturing / analysis system. In addition, a quality control / evaluation apparatus for agricultural products using the color image capturing / analysis system is provided. Furthermore, a method for cultivating agricultural products using the color image capturing / analysis system is also provided.

In order to solve the above problems, the color image capturing / analyzing system of the present invention is:
<1> A color image imaging / analysis system comprising (A) a color image imaging system and / or (B) a color image analysis system,
(A) The color image capturing system has a cylindrical diffused light source, the object is installed in the light source, and the camera is installed on the upper part of the axis perpendicular to the center of the cylindrical bottom surface of the light source .
(B) Color image analysis system, Ri Do from an image analyzing step based on the CDE (Color Distribution Entropy), and color analyzing step based on the Color Histogram,
In the image analysis step, the annular circle including both ends of the major axis of the object image is set as the maximum annular circle, and the number N of the annular circles for suppressing the CDE error is determined .
<2> A biomineral quality control and / or quality evaluation system, wherein the color image capturing / analysis system according to <1> is used.
<3> A quality control and / or quality evaluation system for agricultural products, wherein the color image capturing / analysis system according to <1> is used.
<4> A farming diagnosis apparatus for agricultural products, wherein the color image imaging / analysis system according to <1> is used.
<5> A quality control and / or quality evaluation system for agricultural products according to the above <3>, wherein the CDE similarity, shape similarity, and size similarity with standard agricultural products are used as evaluation indices. It is characterized by.
<6> The above-mentioned <4> agricultural product cultivation diagnosis apparatus, characterized in that a CDE similarity, a shape similarity, and a size similarity with standard agricultural products are used as evaluation indexes.

ADVANTAGE OF THE INVENTION According to this invention, the color image imaging imaging and analysis system which removes or reduces the surface reflection of a target object can be provided. A color image analysis system based on Color Histogram and CDE is also provided.
According to the present invention, it is possible to provide a biomineral quality control / evaluation apparatus using the color image capturing / analysis system. Also, it is possible to provide a quality control / evaluation apparatus for agricultural products using the color image capturing / analysis system. Furthermore, according to the invention, it is possible to provide a method for cultivating agricultural products using the color image imaging / analysis system.

Schematic which shows an example of the color image imaging system of this invention. 1 is a design diagram illustrating an example of a color image capturing system of the present invention. 1 is an external view showing an example of a color image capturing system of the present invention. Visible spectrum of SPIRAL VITALIGHT. The visible spectrum of the present invention of the paper constituting the diffusion cylinder. Illuminance distribution in the diffusion cylinder. A captured image of the object. (A) An image captured without using the color image capturing system of the present invention, and (b) an image captured using the color image capturing system of the present invention. Annular circle setting. (A) Conventional method, (b) The present invention. Model image used to determine N number of annular circles. (B) is reduced by 70% of (a), and (c) is reduced by 50% of (a). The figure which shows the relationship between Color bin and a chromaticity change. The flowchart of an analysis program. Images of pearls classified by grade. Color Histogram of pearls by grade. An image of A-rank pearls taken with a different diffusion cylinder diameter. Strawberry image used to determine N number of annular circles. (B) is 60% reduction of (a), (c) is 150% enlargement of (a). Color image of the strawberry surface. The figure which shows the similarity degree of CDE of a strawberry. The figure which shows the similarity of the shape and magnitude | size of a strawberry. (Upper figure) Shape, (Lower figure) Size. The figure which shows the similarity (CDE, three-dimensional plot of a shape and a magnitude | size) of a strawberry. The figure which shows the similarity (plot of the Euclidean distance from a standard) of a strawberry. The figure which shows the total similarity of the strawberry by 6-dimensional Euclidean distance. The figure which shows the cosine scale in three-dimensional space. The figure which shows the total similarity of the strawberry by a 6-dimensional cosine scale.

The color image capturing / analyzing system of the present invention includes a color image capturing system and a color image analyzing system. It is also possible to use each independently (for example, an image captured by the color image capturing system may be analyzed using an arbitrary image analysis method, or an image captured by an arbitrary apparatus / condition) May be provided to the present color image analysis system). By using a combination of a color image capturing system and a color image analysis system, management / evaluation closer to human eyes becomes possible.

<1. Color imaging system>
The color image capturing system of the present invention mainly includes (1) a sample mounting table, (2) a diffusion cylindrical light source, (3) a camera, and (4) other configurations. FIG. 1 is a schematic diagram of the color image capturing system of the present invention, FIG. 2 is a design diagram, and FIG. 3 is an external view.

(1) Sample setting table The object to be imaged is biological minerals (pearls, etc.) or agricultural products (including strawberries for small items, watermelons and cut items for large items), etc. In addition to being able to adjust the height of the sample, the following (2) is designed so that the cylinder can be opened and closed. The diameter of the cylinder is set to 38 [cm] so that the watermelon can be imaged. For larger objects, this can be accommodated by increasing the diameter of the cylinder.

(2) Diffusing cylindrical light source The diffusing cylindrical light source desirably irradiates the imaging object with substantially uniform light, and includes a light bulb type fluorescent lamp and a cylinder. Although it does not specifically limit as a light bulb type fluorescent lamp which is a light source, it is desirable to have high color rendering properties, and SPIRAL VITALIGHT (LS Co., color rendering index 92, color temperature 5500 [K]) is exemplified. Further, the material of the cylinder is not particularly limited, but it is desirable that the spectral pattern irradiated to the object does not have a significant distribution, and standard white Kent paper is exemplified. Examples of standard white Kent paper include luminescence neutral white (made of special paper), marshmallow COC (made of Oji Special Paper), and the like.
The result of measuring the visible spectrum of SPIRAL VITALIGHT with a spectrophotometer (i1, manufactured by X-Rite) is shown in FIG. Thus, even if it is a light source which has arbitrary spectral patterns, the spectral pattern irradiated to a target object should not have remarkable distributions, such as a peak, through standard white Kent paper (Drawing 5).
By illuminating the cylinder from four points, the object is irradiated with diffused light inside the diffusion cylinder. The number of light sources can be increased or decreased from four points. Of course, the light source itself may have a cylindrical shape by using an LED light source or the like.
Illumination unevenness in the diffusion cylinder measured by a color illuminometer (CL-200, manufactured by Konica Minolta) is shown in FIG. It turns out that it has a substantially uniform illuminance distribution in the cylinder.

(3) Although the camera that performs camera imaging is not particularly limited, a digital single-lens reflex camera is desirable, and D300 (Nikon) and the lens are AF Zoom Nikkor ED (Nikon). In addition, as an imaging stand for fixing the camera, L-4 (manufactured by KING) having a high degree of freedom with respect to changing the height of the camera is exemplified.

(4) Other configuration It is preferable to cover the entire imaging system with a black curtain in order to further block the influence of external light. In order to eliminate the reflection of the light of the dark curtain in the dark room, the double curtain is suitable as the dark curtain where the outer surface is a light shielding first grade dark curtain and the lining is a black dark curtain. In addition, a monitor with high color reproducibility is installed outside the imaging system, and monitoring can be performed by connecting the (3) camera and the monitor. The monitor having high color reproducibility is not particularly limited, but SyncMaster XL24 (manufactured by Samsung) is exemplified.

For iron balls, pearls, tomatoes and eggplants, an image captured without using the color image capturing system of the present invention (FIG. 7a) and an image captured with the color image capturing system of the present invention (FIG. 7b) are compared. As a result, it can be seen that by using the color image capturing system of the present invention, it is possible to capture an image from which the influence of reflected light has been removed even for an object having gloss, uneven color, and unevenness on the surface.

<2. Color image analysis system>
The color image analysis system of the present invention mainly comprises (1) image analysis based on CDE and (3) color analysis based on Color Histogram and CDE. For an object with a complex surface color, distribution, shape, etc., it is desirable to insert (2) an optimal N number determination step for the annular circle between (1) and (3). The color image analysis system is executed on a computer such as a computer.

(1) Image analysis based on CDE
CDE (Color Distribution Entropy) is a technique for obtaining color spatial information in an image based on NSDH (Normalized Spatial Distribution Histogram) described in Non-Patent Document 4 and information entropy described in Non-Patent Document 5.
If Ai is a set of Color bin i, | Ai | is the number of Ai, and Ci is the center of gravity of Ai, N circles with Ci concentric are drawn at equal intervals. At this time, the number of pixels of Color bin i included in the j-th annular circle is assumed to be | Aij |. If Pi is NSDH of Color bin i, Pi is defined as in Equation 1 (Equation 1) and Equation 2 (Equation 2).
If Ei is the CDE of Color bin i, Ei is defined as shown in Equation 3 (Equation 3).
Equation 3 gives how much an arbitrary color is dispersed in the image. The larger the value of Ei, the more dispersed the pixels of Color bin i.
Furthermore, I-CDE (Improved CDE) with improved CDE has also been proposed. In I-CDE, first, an annular circle close to the center of gravity is weighted by using a weighting function (Equation 4 (Equation 4)).
Thereby, Formula 3 becomes Formula 5 (Formula 5).
Also, different histograms may have the same entropy due to entropy symmetry. That is, a completely different image may be expressed with the same entropy. In order to solve this problem, a weight function using a histogram area has been proposed. If H is a histogram {p _1, p _2,..., P _n }, H histogram area A (H) is defined as in Equation 6 (Equation 6).
The weight function g (H) using the histogram area is defined as shown in Equation 7 (Equation 7).
As a result, Equation 3 becomes Equation 8 (Equation 8).
From Equation 3, Equation 5, and Equation 8, I-CDE is defined as Equation 9 (Equation 9).
The above CDE characterizes the entire image. Therefore, when characterizing an image, information entropy is calculated based on an annular circle having a different centroid coordinate and diameter for each color as shown in FIG. There was a problem that would become. In the present invention, as shown in FIG. 8B, the size of the annular circle is determined based on the major axis of the object, and the color distribution information is normalized by the size of the object.

(2) Image analysis based on CDE (determining the optimal N number of annular circles)
When digital image analysis is performed, various errors occur in the process of image acquisition and program calculation, so even if a theoretically derived analysis method is adopted, these errors are not included in the actual color analysis results. Will be included. In particular, when targeting agricultural products, the size of the maximum annular circle determined in accordance with the analysis target and the number of divisions of the annular circle may affect the accuracy of the analysis result.
Therefore, similar images having the same color characteristics, images having the same image size and different color spatial distribution, and images having different base hues as shown in FIG. 9 were created. Table 1 shows the color histogram of the model image. Except for the figure (f), the color balance of red and black is almost the same, and (a), (d), and (e) are different in the spatial arrangement of colors. (A), (b), and (c) are similar images of 100%, 70%, and 50%. Also, (a) and (f) are images whose background images have red and blue hues. Each of these images is subjected to color analysis by changing the number of divisions N of the annular circles into 10 divisions, 20 divisions and 30 divisions, and an error evaluation is performed in the actual image analysis work, and the number N of annular circle divisions and the CDE value are evaluated. Was examined.

Table 2 shows the results of CDE analysis of the model image (Fig. 9). Since FIGS. 9A to 9C are similar images having the same color characteristics, theoretically, when the optimum number N of annular circles corresponding to the complexity of the color distribution of the image is given, Is considered to give a constant CDE value regardless of the size of the image. As shown in Table 2, when the number of divisions was set to 10 or 30, the CDE value variation at i = 15 (red) and i = 36 (black) was confirmed to be 4.1% at maximum. However, if the number of divisions is set to 20, the CDE variation at i = 15 (red) and i = 36 (black) is within 1%. In this model image, N = 20
Was shown to be suitable.
Thus, it can be seen that the number N of annular circles is an extremely important parameter in color analysis using the CDE technique. Therefore, the number N of divisions of the annular circle needs to be determined for each analysis object based on the color feature, and this parameter can be used as an index representing the complexity of the color feature of the image.
Further, when the CDE values in FIGS. 9A, 9D, and 9E having the same image size and different color spatial distribution are compared, i = 15
Although not so large, a difference of 37% or more was confirmed at i = 36. From this result, in the model image of FIG.
If a difference of 1% or more was recognized, it was considered that there was a significant color difference regardless of the size of the object. Further, comparing the results of Color Histogram analysis in FIGS. 9A and 9F with different base hues, the representative color in FIG. 9A is i = 15, whereas FIG. ), I = 174, so the overall color difference of the image can be evaluated from the Color Histogram results.

(3) Color analysis based on Color Histogram and CDE Next, color analysis of the image is performed using Color Histogram and CDE. By using these two analysis methods, Color
By comparing the appearance frequency of colors with Histogram, rough images can be evaluated, and by comparing images with similar Color Histograms with CDE, more detailed images can be evaluated. By converting the color space of the image used for analysis from RGB to HSV and reducing it to 256 colors, the load on the computer is reduced and the analysis results can be easily evaluated. The color after color reduction is represented by a value between 0 and 255.
Let bin i. FIG. 10 shows the relationship between Color bin i and chromaticity change.

By making the above analysis into a program, significant efficiency can be achieved. In consideration of use in various environments, the image analysis program is preferably developed in Java (registered trademark), which can create software independent of the platform. A flowchart of the developed analysis program is shown in FIG.

One preferred embodiment of the present invention will be specifically described below by way of examples. However, the technical scope of the present invention is not limited by the following embodiments, and various modifications may be made within the scope of the present invention. Can be implemented.

<Example 1: Color analysis of pearls>
(1) The pearls ranked A, B, and C by pearl comparison specialists by rank were imaged by a color image imaging system, and color analysis was performed to detect a difference in rank. Table 3 shows the camera settings during pearl image capture.
FIG. 12 shows the captured image, and FIG. 13 shows the analysis result by the color image analysis system. Appearing Color
The number of bins was 40 for A, 27 for B, and 24 for C, and pearls could be classified by grade using the color image capturing / analysis system of the present invention.
(2) Optimal diffusion cylinder FIG. 14 shows a pearl image captured by changing the diameter of the diffusion cylinder. Color that appears when the diameter of the cylinder changes
It was found that the number of bins changed (Table 4). In particular, when the diameter of the cylinder is 37 [cm], the difference in rank is noticeable in the analysis result. Therefore, it can be determined that the cylinder with the diameter of 37 [cm] is optimal for classifying the pearls.

<Example 2: Color analysis of strawberry>
Color images of strawberries were picked up by a color image pickup system, and the difference in variety was detected.
Strawberries as samples were collected from January to February 2010 from 34 kinds of strawberries grown under the three types of cultivation conditions cultivated at the Vegetable Tea Industry Research Institute and stored in the refrigerator. The one within 72 hours after collection was used. Table 5 shows the list of strawberry varieties used, and Table 6 shows the camera settings when capturing strawberry images.

First, a standard strawberry image was selected, and variations of the strawberry size were taken into consideration, and 60% and 150% images were created as two similar images having the same color characteristics as this image. FIG. 15 shows these strawberry images. These images are subjected to color analysis by changing the number of divisions N of the annular circles to 10, 20, 30, 40 and 50, respectively, and error evaluation in actual image analysis work is performed. The determination of the number of annular circles considered to be optimal for the analysis and the error that occurred at that time were examined.
Tables 7 and 8 show the results of color analysis of the standard strawberry image with different magnification rates. Table 8 shows that in the color analysis of standard strawberries, when the number of divisions of the annular circle is 40, the CDE value variation is the smallest within 0.01%, and when the number of divisions is increased or decreased, the error increases. I can confirm.

Next, the surface image of strawberry by a color image analysis system is shown in FIG. In the surface image, it was confirmed that the shining around the species that adversely affects the color evaluation disappeared and the surface texture was recorded accurately.

<Example 3: Cultivation diagnosis of strawberry>
For the color image of the strawberry (Kurume No. 16) imaged by the color image imaging system, the color image analysis system calculates the standard strawberry CDE, shape, and size from the image of all the samples. The degree of similarity of CDE, shape and size with strawberry was calculated.
FIG. 17 shows a calculation method of the strawberry CDE in Formula 10 (Equation 10), and shows the CDE similarity between all samples and standard strawberry.
Next, the shape of the strawberry was calculated by the method described in Non-Patent Document 6. A diagram showing the similarity in shape between all samples and a standard strawberry is shown in the upper diagram of FIG.
Furthermore, the calculation method of the size (major axis and minor axis) of the strawberry is shown in Equation 11 (Equation 11), and a diagram showing the similarity in size between all samples and a standard strawberry is shown in the lower diagram of FIG.
The CDE, shape, and size of all the samples obtained as described above and the standard strawberry were plotted three-dimensionally (FIG. 19), and the Euclidean distance of the similarity from the standard strawberry was obtained for the plots of all the samples ( FIG. 20). By capturing a color image of the strawberry in the cultivation process and comparing it with FIG. 20, the similarity between the strawberry in the cultivation process and a standard strawberry can be detected. That is, the cultivation process can be diagnosed from the similarity between the strawberry in the cultivation process and the standard strawberry.

Furthermore, similarities of CDE, shape and size of all the samples and standard strawberry were obtained for the strawberry cross section and plotted together with the results of FIG. 20 (FIG. 21). The overall similarity based on the 6-dimensional Euclidean distance enables management / evaluation regarding not only the appearance quality but also the interior quality. That is, by taking a color image of a strawberry in the cultivation process and comparing it with FIG. 21, the cultivation process including the internal quality can be diagnosed from the similarity between the strawberry in the cultivation process and a standard strawberry.

The above is an agricultural product management / evaluation system using the similarity based on the Euclidean distance, but instead of the Euclidean distance from the standard strawberry, a cosine scale from the standard strawberry (FIG. 22) may be used.
About the strawberry surface and cross section, the size similarity, color similarity, and shape similarity of all the samples and standard strawberry were plotted, and the cosine scale from standard strawberry was calculated | required. The overall similarity based on the 6-dimensional cosine scale shown in FIG. 23 is more consistent in the permutation of strawberries, indicating effectiveness.

From the above, the color image imaging / analysis system of the present invention arbitrarily selects / extracts parameters such as CDE, size, color, shape, size, etc. in the cultivation process, harvesting process, processing / distribution / sales process of agricultural products. The appearance / introspection quality can be managed and evaluated using the similarity between the captured image and the standard agricultural product image in each process.

Claims (6)

A color image capturing / analyzing system comprising (A) a color image capturing system and / or (B) a color image analyzing system,
(A) The color image capturing system has a cylindrical diffused light source, the object is installed in the light source, and the camera is installed on the upper part of the axis perpendicular to the center of the cylindrical bottom surface of the light source .
(B) Color image analysis system, Ri Do from an image analyzing step based on the CDE (Color Distribution Entropy), and color analyzing step based on the Color Histogram,
A color image capturing / analyzing system characterized in that in the image analysis step, an annular circle including both ends of the major axis of the object image is set as a maximum annular circle, and the number N of divisions of the annular circle that suppresses CDE errors is determined. . A quality control and / or quality evaluation system for biominerals using the color image capturing / analysis system according to claim 1 . A quality control and / or quality evaluation system for agricultural products using the color image capturing / analysis system according to claim 1 . A cultivation diagnostic apparatus for agricultural products using the color image imaging / analysis system according to claim 1 . The quality control and / or quality evaluation system for agricultural products according to claim 3, wherein the evaluation index is a CDE similarity, a shape similarity, and a size similarity with a standard agricultural product. 5. The agricultural cultivation diagnostic apparatus according to claim 4, wherein the CDE similarity, the shape similarity, and the size similarity with a standard agricultural product are used as evaluation indexes.