JPS62194587A - Fruit recognizing device - Google Patents

Abstract

PURPOSE:To obtain position information representing the position of a recognition fruit by sampling the grey level in plural stages and extracting a highlight area where the grey level of sampled variable density picture information is maximum and obtaining the position of its center of gravity. CONSTITUTION:The video signal image picked up by a color video camera 1 is separated into a read signal, a blue signal, a luminance signal, etc., by an NTS decoder 2, and the difference (R-B) between the red signal R and the blue signal B is operated by a color separating circuit 3 and is compared with a set threshold and is converted to binarization color picture information F1 corresponding to only the color of the recognition object fruit. The luminance signal is allowed to pass a gate circuit 4, which is opened only when information F1 is in the high level, and is extracted as variable density picture information corresponding to only the color of the fruit, and this information is sampled as digital variable density picture information is sampled as digital variable density picture information S2 by an A/D converter 5. Information S2 is checked by processors CPU1 and CPU2 with respect to the grey level to extract each highlight area, and the position of the center of gravity is operated to obtain the position information of the fruit.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is directed to, for example, detecting the positional relationship of a fruit to be worked with respect to a fruit work machine and obtaining control information for guiding a work manipulator. Regarding the fruit recognition device used as a means etc., in detail, it is provided with an illumination means for illuminating an imaged object, and an imaging means for taking an image of the imaged object when illuminated by the illumination means, and an identification device corresponding to the recognized fruit from the imaged image information. The present invention relates to a fruit recognition device including fruit position detection means for extracting gray scale image information in a color range and detecting recognized fruit position information based on the gray level of the extracted gray scale image information.

[Conventional technology]

Fruits such as tangerines and apples are generally spherical in shape, so when the recognition target fruit is illuminated, the position that appears as the apex of the spherical shape, that is, the position where the brightness is at its maximum value, reflects the brightest light. becomes. Since the fruit is spherical, the brightest position exists for each of the plurality of objects. Therefore, this position that appears brightest can be treated as representative position information of each fruit to be recognized.

This type of fruit recognition device described above detects positional information of the fruit to be recognized by utilizing the property that when the fruit is illuminated from a predetermined direction, a point representing that position appears the brightest. .

Then, the applicant uses the above characteristics to mask the image information captured by illuminating the recognition target with a specific color corresponding to the target fruit, so that the image information is within the same coordinate range as the masked specific color. We have previously proposed a fruit recognition device that is equipped with means for extracting only certain grayscale image information and detecting the brightness peak position where the brightness of the grayscale image information reaches its maximum value. (Tokugan Sho 6
(See No. 0-128933) [Problems to be Solved by the Invention] However, the peak position of the brightness is detected from the grayscale image information in the range corresponding to the above-mentioned specific color, and the detected peak position information is used to identify the recognized fruit. The configuration in which position information is used has the following disadvantages, and there is room for improvement.

That is, it is rare for the surface of a fruit to have a smooth spherical shape, and usually there are small depressions due to fine irregularities or scratches.

Therefore, when a fruit with locally varying irregularities on its surface is illuminated, the brightness will locally change in the irregularities.

On the other hand, in a device that involves this type of image processing, analog image signals from imaging means such as a camera or a camera are digitized and processed, and quantization errors and sampling errors occur due to the digitization. In order to reduce these errors, it is possible to increase the resolution of quantization and sampling, but if this is done, the number of pixels that make up the digitized image information will be extremely large, making it difficult to achieve practical processing speeds. In addition, if the number of pixels in the image information is large, even subtle changes in brightness will be recognized, and the brightness peak position will appear as if there were multiple locations on the surface of a single fruit. However, there is also the disadvantage that recognition accuracy may be lower.

Therefore, when extracting the above-mentioned grayscale image, it is possible to filter the image signal to remove small changes in brightness, but since the size and shape of the fruit are uncertain, it is difficult to remove them with a simple filter. It's something that doesn't exist. or,
When detecting the brightness peak position mentioned above, it is possible to provide a mask for detecting the peak position and detect the peak position within the mask, but the size and shape of the fruit are uncertain. Therefore, the mask for image processing cannot be made unnecessarily large.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to extract the brightest part from the grayscale image information in a range masked with a specific color corresponding to the recognition-compatible fruit. The purpose is to avoid being affected by local changes in brightness when detecting fruit position information.

[Means for solving problems]

The characteristic configuration of the fruit recognition device according to the present invention includes a sampling means for sampling the gray level of the gray level image into a plurality of stages in order to extract the gray level image information of the specific color range; Maximum Highlight) 61
The present invention is provided with a 61-area extracting means for extracting a highlight area, and a center-of-gravity detecting means for detecting the center of gravity of the extracted highlight area, and their functions and effects are as follows.

[For production]

In other words, when extracting grayscale image information corresponding to a specific color, the gray level of the grayscale image information is sampled in multiple stages, and the highlight area where the gray level of the sampled grayscale image information is maximum is extracted. By doing this, the area containing the maximum value of brightness, that is, the point representing the brightest visible fruit position, is identified, and by finding the center of gravity, one piece of position information representing the position of the recognized fruit is obtained. .

〔Effect of the invention〕

In other words, by extracting the highlight area where the gray level of the gray image information sampled in multiple stages is maximum, the area containing the brightest visible position of the recognized fruit is identified in a narrow range, and its center of gravity is recognized. Since the information is based on the position information of the fruit, even if there is a local change in brightness within the highlight area where the gray level is maximum, the influence of this change can be reliably removed. In addition, since the center of gravity of the highlight area specified in this narrow range is detected, the amount of image information processing data for detecting the center of gravity position is small, and the processing speed for detecting the center of gravity position becomes an issue. There isn't.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

FIG. 1 is a block diagram of a fruit recognition device that obtains positional information corresponding to each fruit to be recognized from images captured outdoors. By subtracting the blue component color signal (CB) from the red component color signal (R) that makes up the color image, it is possible to recognize colors other than fruit colors, leaves, sky, etc. The binarized image information (F1) corresponding only to the fruit color (C) obtained by removing the color component corresponding to the background object is extracted, and this binarized image information (F1) is used to generate the luminance signal ( The brightness image information (S+) in a specific color range corresponding only to the color of the fruit (C) is extracted by masking the color (Y), and the brightness, that is, the gray level, is sampled in multiple stages to extract the brightest highlight area ( P0) and find the centroid position (Pt) of the highlight area (P0) to obtain positional information for each of the multiple fruits that appear to be overlapped in a separated state. .

That is, a color video camera (1) as an imaging means
The NTSC format color video signal (S0), which is output as image information (see Figure 3 (a)) captured by the
), horizontal synchronization signal (HI1), red signal (
R), a blue signal (B), and a luminance signal (Y) as grayscale image information, and the color separation circuit (3)
Therefore, the difference (R) between the red signal (R) and the blue signal (B) is
-B) is compared with the set threshold value (Cref), and the binarized color image information (F1) (F1) is the binarized image information of the color corresponding only to the color (C) of the fruit to be recognized. Figure 3 (r
1)).

The luminance signal (Y) is the binary color image information (F1)
This binary color image information (P
1) is at the "H° level, that is, the gate circuit (4) is opened only in the section corresponding to the fruit color (C). By using the binarized color image information (F1) as a mask, the recognition target It is extracted as grayscale image information (S1) corresponding only to the color (C) of the fruit.Next, this grayscale image information (S1) is converted into 4-bit/1 pixel data by an A/D converter (5). Quantized with resolution 32 x 32
Digital grayscale image information (S2
) (see FIG. 3(c)) and temporarily stored in the image memory (7) via the data buffer (6).

The process of obtaining the binarized color image information (F1), and the process of obtaining the binarized color image information (the luminance signal (Y) using Fυ as a mask)
The process of extracting grayscale image information (S1) corresponding only to the color (C) of the recognition target fruit from , A/D converting it, and storing it in the image memory (7) as digital grayscale image information (S2) for one screen is as follows. The processing operation is controlled by a timing circuit (8) so that the processing operations are performed simultaneously and in parallel in synchronization with the vertical synchronization signal (VD) and the horizontal synchronization signal (HD). Also, a strobe control circuit activated by a control signal from the timing circuit (8) so that a strobe device (10) serving as an illumination means emits light in synchronization with the imaging operation by the camera (1). According to (9), the light emission timing and the light emission intensity are controlled.

Further, the digital grayscale image information (S2) stored in the image memory (7) is transmitted to the CRT controller (11) and T
It is converted into a composite video signal by a V signal generator (12) and displayed on a monitor television (13).

The digital grayscale image information (S2) is then processed by a control processor (CPU, '') and a numerical calculation processor (
CPlh), check the brightness degree, that is, the gray level, at each coordinate point (x, y) and extract each highlight area (Pa) (see Fig. 3 (*)) of the digital grayscale image information (S2). Then, the coordinates (Xj, Yi) of the center of gravity position (p+) are calculated and used as the position information of the recognized fruit.Then, the number of recognized fruits and their position coordinates (Xj, Y
t) as a recognition result, the interface device (
15), it is transmitted to the host computer (CPU 1) and the overall operation is controlled. still,
In FIG. 1, (16) represents each processor (CPU1)
, (cpoz) and calculation data, and various data exchanged with the host computer (CPU0).

Hereinafter, the operation of the fruit recognition device will be described in detail based on the flowchart shown in FIG.

That is, the grayscale image information (S1) of a specific color (C) range obtained by masking the luminance signal (Y) with the binary color image signal (F1) is sampled and stored in the image memory (7). By doing so, image information corresponding only to the fruit to be recognized is extracted (step 1l1).

Next, the black and white of the digital grayscale image information (S2) stored in the image memory (7) is inverted and converted into inverted image information (revision) (see Figure 3 (=)), and labeling processing is performed. By erasing areas with small areas, areas that appear to have holes due to extreme highlights caused by scratches on the fruit or unevenness on its surface are filled in (step #2 to step 14).

Then, ``The image information with the holes filled in is relabeled, the maximum value of brightness for each label is determined, and the highlight area (p0) with a brightness of the maximum value and the maximum value - 1 is obtained.
(See Figure 3 (N)) and remove the other areas to extract the highlight area (P0) where the gray level is maximum (Step #5 to Step #7)
So, what about this step #5? , in #7,
This constitutes highlight area extraction means. In addition, in the process of extracting this highlight area (P0), extracting the maximum value of brightness and the area with the maximum value - 1 is a relatively rough sampling of image information. This is to prevent large errors in the L71 IIA 8I region due to local brightness changes.

Furthermore, by labeling the extracted highlight region (P0) and calculating the position (Xj, Yi) of the center of gravity position (P1) for each label, position information for each recognized fruit is obtained. There is (S? F #8. Steph Ka9)
). This constitutes a center of gravity position detection means.

Therefore, according to the present invention, as shown in FIG. 3(a),
Not only can the center of gravity of multiple fruits that appear to overlap each other be detected for each fruit, but if the overlap is separated by a distance in the far and near direction, as shown in Figure 4 (a), it is possible to detect the center of gravity of each fruit. Since the fruit on the front side looks the brightest, only the fruit on the front side is recognized as shown in the same figure (T1), and the fruit that is easier to guide the work manipulator is recognized first. The Rukoto. Therefore, if this detected position information is used as control information for guiding a work manipulator such as a fruit harvester, work can be started from an easy-to-work area, improving work efficiency. Furthermore, as shown in Figure 5 (a), even if a part of the fruit appears to be missing due to leaves, etc., the brightest visible part is considered to be approximately in line with the center of gravity of the fruit. Therefore, even a fruit that looks significantly different from its original shape can be compared to t'0 in the same figure.
), it can be recognized accurately.

[Brief explanation of drawings]

Drawing number i shows an embodiment of the fruit recognition device according to the zero invention, and FIG. 1 is a block diagram showing the configuration of the fruit recognition device, and FIG.
The figure is a flowchart showing the operation, and Figures 3 (a) to (
D) is an explanatory diagram of the processed image, and Fig. 4 ((), (II+
) is an explanatory diagram of fruits overlapping in the perspective direction, Figure 5 (a)
, (0) is an explanatory diagram of a fruit that appears partially chipped. (1)... Imaging means, (10)... Illumination means, (S0)... Captured image information, (S1)
..... Grayscale image information, (S2) .... Sampling image information, (P0) ..... Highlight area, (
p+)・・・・・・Center of gravity position.

Claims (1)

[Claims] The apparatus includes an illumination means (10) for illuminating an object to be imaged, and an imaging means (1) for taking an image of the object when illuminated by the illumination means (10), and corresponds to a recognition fruit (F) from the imaged image information (S_0). Shade image information (S_
1) and detects recognized fruit position information based on the gray level of the extracted grayscale image information (S_1), the fruit recognition device comprises fruit position detection means for extracting grayscale image information (S_1) of the grayscale image information (S_1) of the specific color range. S_1), a sampling means (5) samples the gray level in multiple stages, and extracts a highlight area (P_0) where the gray level of the sampled gray image information (S_2) is maximum. a highlight region extracting means for extracting the extracted highlight region (
A fruit recognition device equipped with a center of gravity position detection means for detecting a center of gravity position (P_1) of a fruit (P_0).