JPH0819333A - Device for identifying picture image of cucumber - Google Patents

Abstract

PURPOSE:To clearly and simply obtain a binary picture image from the image- identification device of a harvest robot to harvest cucumbers. CONSTITUTION:Signals outputted from a CCD camera 26, etc., are connected to an image brightness difference means 36 for outputting a color difference signal obtained by deducting a red color signal from a green color signal among the green color signal, the red color signal and a blue color signal inputted from a camera means 37. The image brightness difference means 36 is connected to an image low brightness means 38, which converts a low brightness part to a brightness of zero to give a low brightness color difference signal. The image low brightness means 38 is connected to an image-smoothing means 39 and a picture distance-computing means 40. The image-smoothing means 39 smoothes the inputted color difference signal and is connected to the picture distance-computing means 40. The picture distance-computing means 40 outputs the color difference signal obtained from the image brightness difference means 36 and a computed picture obtained by deducting the smoothened color difference signal obtained from the image-smoothening means 39.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image identification device provided in a harvesting robot or the like, and subtracts a red signal from a green signal among green, red and blue color signals obtained from a color CCD camera. The present invention relates to a cucumber image identifying device for recognizing a cucumber by obtaining a binary image from the color difference signal.

[0002]

2. Description of the Related Art Conventionally, an image identification device such as a harvesting robot images a harvested object such as a cucumber with a monochrome CCD camera. Then, the image signal obtained by the image pickup by the monochrome CCD camera is binarized to obtain a binarized image, and the contour of the cucumber is discriminated from the binarized image. From the contour of the cucumber, the cucumber and the cucumber other than the cucumber, for example, leaves are distinguished to recognize the cucumber.

[0003]

[Problems to be Solved by the Invention] However, monochrome CC
In the binarized image obtained by binarizing the image signal of the D camera, it is difficult to recognize the contour of the cucumber because the colors of the cucumber and the leaves are in the same system. Therefore, a malfunction may occur in the harvest of the cucumber by the harvest robot.

In the present invention, research has been conducted to solve the defects that cannot be solved by such a monochrome CCD camera by using a color CCD camera.

First, a cucumber and leaves as shown in FIG.
A case will be described in which an image is picked up by the color signal of the green signal, the red signal and the blue signal by the color CCD camera at each of the upper measurement position (a), the middle measurement position (b) and the lower measurement position (c). The luminance of the green signal, red signal, and blue signal at each measurement position (a, b, and c) in this case is shown in FIGS. 2 to 4. Here, in FIGS. 2 to 4,
The vertical axis represents the luminance and the horizontal axis represents the pixel position of the color CCD camera. Then, the alternate long and short dash line indicates the green signal, the solid line indicates the red signal, and the broken line indicates the blue signal. Further, a indicates a cucumber, b indicates a leaf (front), c indicates a leaf (back), and d indicates a cucumber shadow.

As a result of this measurement, as shown in FIGS. 2 to 4, at the position of the cucumber a, the brightness of the green signal is the highest,
It can be seen that the red signal and the blue signal are lower in that order.

As shown in FIGS. 5 to 7, a plurality of samples different from those in FIG. 1 are used for each part of cucumber, leaf (front) and leaf (back), and a color difference signal obtained by subtracting a red signal from a green signal. That is, a green signal-red signal (see FIG. 5), a color difference signal obtained by subtracting a blue signal from a green signal, that is, a green signal-blue signal (see FIG. 6), a color difference signal obtained by subtracting a blue signal from a red signal, that is, a red signal -Measure the luminance of each color difference signal of the blue signal (see Figure 7). Here, in FIGS. 5 to 7, the vertical axis represents average luminance and the horizontal axis represents the number of samples. The broken line indicates the cucumber, the solid line indicates the leaf (front), and the alternate long and short dash line indicates the leaf (back). Further, in FIGS. 5 to 7, the data is not the data of each color difference signal in the same sample.

As a result, as shown in FIGS. 6 and 7, the average luminance of each color difference signal of the green signal-blue signal and the red signal-blue signal has the highest average luminance at the position of the cucumber among all the samples. Not that. However, as shown in FIG. 5, in the green signal-red signal, it was measured that the average luminance at the position of the cucumber was higher than the luminance at the front and the back of the leaf. From this, it is understood that it is effective for the extraction of the cucumber to perform the image processing by using the green signal-red signal in which the average brightness of the cucumber is higher than the position other than the position other than the cucumber.

An object of the present invention is to provide a cucumber image identification device based on the above new knowledge, which clearly and easily obtains a binarized image of a cucumber and performs image processing.

[0010]

In order to solve the above-mentioned problems, a first aspect of the present invention is to provide an image pickup means having a color CCD camera for outputting a green signal, a red signal and a blue signal, and a green signal and a red color of the image pickup means. Image brightness difference means for subtracting the red signal from the green signal among the signal and the blue signal, an image low brightness means for reducing the brightness of the color difference signal obtained by the image brightness difference means, and a low brightness obtained by the image low brightness means. An image smoothing unit that smoothes the luminance color difference signal, an inter-screen calculation unit that subtracts the smooth color difference signal of the image smoothing unit from the low-luminance color difference signal of the image low-luminance unit, and an operation screen obtained by the inter-screen calculation unit. And binarizing means for binarizing.

The second aspect of the invention is to provide an image pickup means equipped with a color CCD camera for outputting a green signal, a red signal and a blue signal, and a green signal and a red signal among the green signal, the red signal and the blue signal of the image pickup means. Image brightness summing means for adding, an image low brightness means for lowering the brightness sum color signal obtained by the image brightness summing means, and an image smoothing for smoothing the low brightness color signal obtained by the image low brightness means Means, a luminance ratio means for taking a ratio of the smoothed color signal of the image smoothing means and the green signal of the image pickup means, and a binarizing means for binarizing the luminance colorimetric signal obtained by the luminance ratio means, Is provided.

[0012]

With the above arrangement, in the first invention, the image brightness difference means is configured so that the green signal, the red signal inputted from the image pickup means,
The color difference signal obtained by subtracting the red signal from the green signal in the blue signal is output. The image low-luminance means converts the color-difference signal into a luminance of 0 in the low-luminance portion to obtain a low-luminance color-difference signal. The image smoothing means smoothes the low-luminance color difference signal to obtain a smoothed color difference signal. Then, the inter-screen calculation means performs the inter-screen calculation for subtracting the smoothed color difference signal from the low luminance color difference signal. The operation image thus obtained by the inter-screen operation means is binarized by the binarization means to obtain a binarized image.

According to the second aspect of the invention, the image luminance summing means obtains a luminance sum signal obtained by adding the red signal to the green signal among the green signal, the red signal and the blue signal input from the image pickup means.
The image low-luminance means outputs the luminance sum signal to the low-luminance portion with the luminance 0.
To obtain a low luminance color signal. Then, the image smoothing means smoothes the low-luminance color signal to obtain a smoothed color signal.
Next, the image brightness ratio means obtains a brightness ratio image in which the smoothed color signal and the green signal are ratioed by the brightness ratio means. This brightness ratio image is binarized by a binarizing means to obtain a binarized image.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the first invention will be described with reference to FIGS.

Reference numeral 10 in FIG. 8 is a cucumber image identifying device 11.
1 shows a harvesting robot equipped with. The harvesting robot 10 has
The traveling unit 12 is provided, and a pair of left and right wheels 13, 13 are provided on the front and rear portions of the traveling unit 12.

A front cover 14 is disposed in front of the traveling unit 12, and the front cover 14 has a traveling unit controller 15 for controlling the traveling unit 12, a harvesting unit controller 17 for controlling the harvesting unit 16, and a cucumber image identifying device. The recognition device controller 18 for controlling the controller 11 and the main controller 19 for controlling these controllers are provided. Then, the main controller 19 is connected to the traveling unit controller 15, the harvesting unit controller 17, and the recognition device controller 18 (see FIG. 9).

A conical swivel cover 20 is arranged behind the front cover 14, and a manipulator 21 for gripping and picking cucumbers is mounted on the swivel cover 20. The manipulator 21 includes a first joint 22, a second joint 23, and a third joint 24.
And are connected by arms. 3rd joint 2
The hand portion 25 is arranged at the end of 4. Manipulator 2
A color CCD camera 26 is arranged in the vicinity of 1.

A carrier supply device 27 is arranged at the rear of the traveling unit 12, and the carrier supply device 27 stores a plurality of carriers for carrying harvested cucumbers. An operation device for operating and controlling the harvest robot 10 is provided on the rear side of the carrier supply device 27.

The manipulator 21, the swivel cover 20, and the carrier supply device 27 constitute the harvesting section 16. Next,
A functional block diagram of the cucumber image identification device 11 of this embodiment will be described with reference to FIG.

As shown in FIG. 10, the recognition device control is performed.
The laser 19 connects the OCR 30 to the drive pulse generation circuit 31, and the drive pulse generation circuit 31 is connected to the color CCD camera 26.
Generate the signals necessary to activate the. The drive pulse generation circuit 31 is connected to the driver 32, and the driver 32 is connected to the color CCD camera 26 via the receiver 33. The color CCD camera 26 is connected to the signal processing circuit 34, and the signal processing circuit 34 converts the signal captured by the color CCD camera 26 into a video signal. The signal processing circuit 34 is connected to the A / D conversion circuit 35, and the A / D conversion circuit 35 converts the video signal into a digital signal.

The A / D conversion circuit 35 includes the image brightness difference means 3
6, the image brightness difference means 36 selects the green signal (G) among the green signal (G), the red signal (R) and the blue signal (B) input from the image pickup means 37 including the color CCD camera 26 and the like. The color difference signal obtained by subtracting the red signal (R) from G) is output. The image brightness difference means 36 is an image low brightness means 38.
The image low-luminance means 38 converts the low-luminance portion to luminance 0 to obtain a low-luminance color difference signal. The image low brightness means 38
The image smoothing means 39 and the inter-screen calculation means 40 are connected.
The image smoothing means 39 smoothes the input color difference signal to obtain a smoothed color signal, and connects it to the inter-screen calculation means 40. The inter-screen calculation means 40 outputs a calculated image from which the low-luminance color difference signal obtained by the image low-luminance means 38 and the smoothed color difference signal obtained by the image smoothing means 39 are subtracted.
The inter-screen calculation means 40 is connected to the binarization means 41, and the binarization means 41 binarizes the calculation image obtained by the inter-screen calculation means 40. The image brightness difference means 36, the image low brightness means 38, the image smoothing means 39, the inter-screen calculation means 40 and the binarization means 41 are connected to the calculation means 42, respectively.

The calculation means 42 is connected to the timing circuit 43, and the timing circuit 43 sets the light emission timing of the strobe 44 based on the video signal. Timing circuit 43
To the strobe drive circuit 45, and the strobe drive circuit 4
5 drives the strobe 44. A halogen lamp is used as the strobe 44.

Next, the operation of the first embodiment of the invention will be described with reference to the flowchart of FIG.

The cucumber and the front surface (partially the back surface) of the leaves as shown in FIG. 1 are imaged by the color CCD camera 26 and recognition of the cucumber is started (step S1). Color CCD
A green signal (G), a red signal (R) and a blue signal (B) are sent from the camera 26 to the recognition device controller 19,
It is input to the image brightness difference means 36 via the signal processing circuit 34 and the A / D conversion circuit 35. Here, an image obtained by inverting the green signal (G) of the input cucumber and leaves is shown in FIG. 12 (step S2).

The image brightness difference means 36 calculates the brightness of the color difference signal of the G image-R image, and the image by this color difference signal is A
0 is obtained (step S3). The image low-luminance unit 38 performs low-luminance conversion on the image A0, and converts the low-luminance portion (about 20 or less) of the image A0 to luminance 0. And in FIG.
The image A of the low-luminance color-difference signal having undergone the low-luminance conversion as shown in FIG.
1 is obtained (step S4).

Further, the low-luminance color difference signal input from the image low-luminance means 38 is smoothed by the image smoothing means 39 in the area of horizontal pixels 3 × vertical pixels 3 to obtain an image A2 from the smoothed color difference signal (step S5). ). Then, the inter-screen calculation means 40 performs the inter-screen calculation for subtracting the image A2 from the image A1 to obtain the image A3 (step S6). After that, the binarizing means 41 binarizes the image A3 with the threshold value 1,
The clear binarized image of the cucumber shown in FIG. 14 is obtained (step S7).

Then, the steps S1 to S
7 is repeated (step S8).

With the above-described structure, the cucumber recognition algorithm can be simplified with a clear binary image, so that image processing can be performed at high speed and clear cucumber recognition can be performed.

Since a general-purpose camera can be used for the image processing camera, the system cost can be reduced and the system reliability can be improved.

Next, the embodiment of the second invention will be described with reference to FIG.
Description will be given with reference to FIG. 16, the same components as those of the first embodiment will be designated by the same reference numerals, and the description thereof will be omitted.

In the recognition device controller 46 of the second embodiment of the present invention, the CC which is the image pickup means 37 for picking up an image of a cucumber
The D camera 26 includes the signal processing circuit 34 and the A / D conversion circuit 3
5 to the image brightness summing means 47. Also, A
The / D conversion circuit 35 is connected to the image brightness ratio means.

The green signal (G), the red signal (R) and the blue signal (B) are input to the image brightness summing means 47, and the image brightness summing means 47 adds the red signal (R) to the green signal (G). The luminance sum color signal is output to obtain a C1 image. The image brightness summing means 47 is connected to the image low brightness means 38, and the image low brightness means 38 converts the low brightness part to brightness 0 to obtain a low brightness color difference signal. The image low brightness unit 38 is connected to the image smoothing unit 39. The image smoothing means 39 is an image brightness ratio means 48.
The image brightness ratio means 48 is connected to the binarization means 41. The image brightness ratio means 48 obtains a C2 image from the ratio between the smoothed color signal C1 obtained from the image smoothing means 39 and the green signal (G). Then, the image low brightness means 38, the image smoothing means 39, the image brightness ratio means 48, and the binarization means 41 are connected to the arithmetic means.

Next, the operation of the second embodiment of the invention will be described.

The front surface (partially the back surface) of the cucumber and the leaves surrounded by the frame of FIG. 1 is imaged by the color CCD camera 26 to start recognition of the cucumber (step S10). A green signal (G), a red signal (R) and a blue signal (B) are sent from the color CCD camera 26 to the recognition device controller 46, and the image brightness summing means is passed through the signal processing circuit 42 and the A / D conversion circuit 35. It is input to 47 (step S11). The image brightness summing means 47 obtains the image C1 from the red signal (R) and the green signal (G) (step S12).

Then, the image low-luminance means 38 performs low-luminance conversion on the image C1, and the low-luminance portion (about 20 or less) of the image C1 is converted to 0 (step S13). The image smoothing unit 39 smoothes the low-luminance color signal input from the image low-luminance unit 38 in the area of horizontal pixels 3 × vertical pixels 3 to obtain an image C2 (step S14). The image luminance ratio means obtains an image C3 from the luminance ratio color signal of the image C2 and the green signal (G) (step S15). After that, the binarizing means 41 binarizes the image C3 using the brightness of 15 as a threshold value, and a clear binarized image of cucumber shown in FIG. 14 is obtained (step S16). Then, the step S1
Step 0 to step S16 are repeated (step S17).

Since the cucumber recognition algorithm is simplified by taking the ratio of the image C2 and the green signal as described above, the cucumber recognition can be performed at high speed and easily. Also,
As with the first embodiment of the invention, a general-purpose camera can be used for the image processing camera, so that the system cost can be reduced and the system reliability can be improved.

By the way, the strobe 44 halogen lamp used in the first invention and the second invention has a characteristic that the spectral characteristic also changes when the applied voltage changes. Therefore, color CC
The D camera 26 images a color sample corresponding to the color of the appearance of the cucumber with different applied voltages. This is shown below in FIG.
This will be described with reference to FIGS.

FIG. 17 shows a color sample, in which the first to sixth color samples are arranged in order in the vertical direction from the top. In addition, the systematic colors 9-1 to 10-2 are shown in the horizontal direction. 18,
In FIG. 19, the vertical axis represents luminance, and the horizontal direction represents pixel positions 1 to 6 of the color sample from the left. 20 to 2
2, the vertical axis indicates the normalized luminance difference, and the horizontal direction indicates the systematic color 1
The color samples from No. 1 to No. 6 of 0-1 are shown. Here, the normalized luminance difference means a voltage difference between the green signal, the red signal, and the blue signal, which is obtained by imaging the color sample shown in FIG. 17 with a CCD camera.

Next, by applying different voltages to the color samples No. 1 to No. 6 of the system color 10-1 shown in FIG. 17, the color CCD camera 26 takes an image along the luminance measurement line, and the green signal ( G), the brightness of the red signal (R) and the blue signal (B) are measured (see FIGS. 18 and 19). The system color 10-1 has six color swatches arranged from dark green to light green in order from the top.

FIG. 18 shows the luminances of the green signal (G), the red signal (R) and the blue signal (B) obtained by imaging the color sample with an applied voltage of 10V. Similarly, FIG. 19 shows an image obtained by applying an applied voltage of 14V. As shown in FIGS. 18 and 19, of the green signal (G), the red signal (R), and the blue signal (B) obtained by imaging the color sample, the green signal (G) has the highest luminance.

Next, FIG. 20 shows the normalized luminance difference between the green signal (G) and the red signal (R) when the applied voltage is 10 V and 14 V. Similarly, the normalized luminance difference between the green signal (G) and the blue signal (B) is shown in FIG. 21, and the normalized luminance difference between the red signal (R) and the blue signal (B) is shown in FIG.

From these FIGS. 20 to 22, the applied voltage is 1
It can be seen that the normalized luminance difference between the green signal (G) and the red signal (R) is high when the voltage is 4V. From this, it can be expected that a clear binary image can be obtained by setting the applied voltage to 14V.

Next, in the second system color 10-1 of the color sample, the normalized luminances of the green signal (G) and the red signal (R) in the case of imaging with applied voltages of 10 V and 14 V are compared ( (See FIG. 23).

It is shown that the average value of the standardized luminance of the green signal (G) -red signal (R) is 14V higher than 10V. Therefore, although it can be estimated that 14V is effective, two average values of the applied voltage of 10V and 14V are further tested. In this case, when it is examined whether the applied voltages 10V and 14V are equal in dispersion, the second system color 10-1 has different dispersion.

Therefore, the Welch's test is used to check whether the average values of the applied voltages of 10 V and 14 V are equal. The statistic t becomes 15.4371, and the degree of freedom is 56.8694. The value of the statistic t is much higher than the value of the 99% confidence interval in the t-distribution table. Therefore, it cannot be said that the average values are the same as the Welch's test value. Therefore, as shown in the average column of FIG. 23, it can be estimated that the average value of the standard luminance difference of 14V of the applied voltage is higher than the average value of the standard luminance difference of 10V of the applied voltage.

Similarly, in the color samples of the other system colors 10-1 as well, it was confirmed that the average value of the standardized luminance of 14V of the applied voltage is higher than the average value of the standardized luminance of 10V of the applied voltage. did it.

As described above, the higher the applied voltage, the higher the brightness of the green signal-red signal. By utilizing this fact, the applied voltage is set higher by the strobe drive circuit 45 which constitutes the recognition device controllers 19 and 46.

Therefore, the color difference signal of green signal-red signal is increased by increasing the voltage applied to the light source such as a halogen lamp, and a clear binary image can be obtained. Therefore, the cucumber recognition algorithm is simplified by the color difference signal of the green signal-red signal, and the cucumber can be recognized easily at high speed.

[0049]

As described above, in the first and second inventions, even if the colors of the cucumber and the leaves are of the same system, a clear binarized image can be obtained, so that the recognition algorithm for recognizing the cucumber can be obtained. The image processing can be executed at high speed, and the cucumber can be easily recognized.

Further, since the camera for image processing uses a general-purpose product, the system cost can be reduced and the system reliability can be improved.

[Brief description of drawings]

FIG. 1 is a diagram showing a state of cucumbers and leaves recognized by a cucumber image identifying device of a first invention.

FIG. 2 is a diagram showing the brightness of a green signal, a red signal, and a blue signal when the cucumber shown in FIG. 1 is imaged at the upper measurement position (a).

FIG. 3 is a diagram showing the brightness of a green signal, a red signal, and a blue signal when the cucumber shown in FIG. 1 is imaged at a measurement position (b) in the middle stage.

FIG. 4 is a diagram showing the luminance of a green signal, a red signal, and a blue signal when the cucumber shown in FIG. 1 is imaged at the lower measurement position (C).

FIG. 5 is a diagram showing an average luminance of green signal-red signal when a cucumber, a leaf (front) and a leaf (back) are imaged.

FIG. 6 is a diagram showing an average luminance of green signal-blue signal when a cucumber, a leaf (front) and a leaf (back) are imaged.

FIG. 7 is a diagram showing an average luminance of a red signal-a blue signal when a cucumber, a leaf (front) and a leaf (back) are imaged.

FIG. 8 is a rear view of the harvesting robot used in the embodiment of the first invention.

FIG. 9 is a block diagram showing a control relationship of the entire harvesting robot.

FIG. 10 is a functional block diagram of a recognition device controller of the cucumber image recognition device of the first invention.

FIG. 11 shows a flowchart of the first invention.

FIG. 12 shows an inverted image of the green signal of FIG.

13 is a diagram in which the low-luminance portion of the green signal-blue signal of FIG. 1 is removed.

FIG. 14 is an image after the binarization process of FIG.

FIG. 15 is a functional block diagram of a recognition device controller of the cucumber image recognition device of the second invention.

FIG. 16 shows a flowchart of an embodiment of the second invention.

FIG. 17 shows a color sample.

FIG. 18 is a diagram showing the luminance of a green signal, a red signal, and a blue signal when an applied voltage of 10 V is applied to the color sample shown in FIG.

19 is a diagram showing the luminance of a green signal, a red signal, and a blue signal when an applied voltage of 14 V is applied to the color sample shown in FIG.

20 is a diagram illustrating an applied voltage of 10 V and 14 of the color sample illustrated in FIG.
It is a figure which shows the normalization brightness difference of the green signal-red signal in V.

FIG. 21 is an applied voltage of 10 V and 14 of the color sample shown in FIG.
It is a figure which shows the normalization brightness difference of the green signal-blue signal in V.

22 is a diagram illustrating an applied voltage of 10 V and 14 of the color sample illustrated in FIG.
It is a figure which shows the normalization luminance difference of the red signal-blue signal in V.

FIG. 23 is a diagram showing statistical values of a green signal-blue signal at applied voltages of 10 V and 14 V in the second stage of the color system 10-1 of the color sample.

[Explanation of symbols]

 19, 46 Recognition device controller 34 Signal processing circuit 35 A / D conversion circuit 36 Image brightness difference means 37 Imaging means 38 Image low brightness means 39 Image smoothing means 40 Inter-screen operation means 41 Binarization means 42 Operation means 47 Image Brightness Sum Means 48 Image Brightness Ratio Means

Claims (2)

[Claims] 1. An image pickup means having a color CCD camera for outputting a green signal, a red signal and a blue signal, and an image brightness difference obtained by subtracting the red signal from the green signal among the green signal, the red signal and the blue signal of the image pickup means. Means, an image low-luminance means for reducing the luminance of the color difference signal obtained by the image luminance difference means, an image smoothing means for smoothing the low luminance color difference signal obtained by the image low luminance means, and an image low luminance means And a binarizing unit for binarizing the arithmetic screen obtained by the inter-screen arithmetic unit, the inter-screen arithmetic unit subtracting the smooth color difference signal of the image smoothing unit from the low-luminance color difference signal. Cucumber image identification device. 2. An image pickup means equipped with a color CCD camera for outputting a green signal, a red signal and a blue signal, and a sum of image luminances to which a green signal and a red signal are added among the green signal, the red signal and the blue signal of the image pickup means. Means, an image low-luminance means for reducing the luminance of the luminance sum color signal obtained by the image luminance summing means, an image smoothing means for smoothing the low luminance color signal obtained by the image low-luminance means, and an image smoothing means. Smoothing color signal of the conversion means,
A cucumber image identification device comprising: a luminance ratio means for taking a ratio of a green signal of the image pickup means; and a binarization means for binarizing the luminance colorimetric signal obtained by the luminance ratio means.