JPH0414725B2 - - Google Patents

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is directed to, for example, detecting the positional relationship of a fruit to be worked with respect to a fruit work machine and obtaining control information for guiding a work manipulator. A fruit recognition device used as a means, etc., specifically, determines an area corresponding to a recognition target fruit color set from color image information out of image information captured by an imaging means, and converts it into binarized color image information. a color binarization means for extracting; an edge image detection means for determining a portion of the captured image information in which a change in brightness is larger than the grayscale image information and extracting it as binarized edge image information; The present invention relates to a fruit recognition device that obtains binarized image information in which a region corresponding to the fruit to be recognized is separated and extracted from the captured image information by being provided with a subtraction means for subtracting the binarized edge image information from the binarized color image information. .

[Conventional technology]

This type of fruit recognition device described above is able to separate the fruits into individual fruits even when the fruits are crowded together or overlapped so that they appear as a single cluster. This makes it possible to obtain binary image information.

By the way, in order to obtain the binarized edge image information by the edge image detection means, it is conceivable to input the grayscale image information of the captured image information as it is to the edge image detection means.

[Problem that the invention seeks to solve]

However, if the grayscale image information of the captured image information is used as is to obtain the binarized edge image information, the processing is time-consuming and has the disadvantage of making it difficult to perform fruit recognition at high speed. .

In other words, the edge image detection means, for example, performs differential processing on the grayscale image information of the captured image information,
After determining the value and direction of the brightness change, and removing noise information such as isolated edge points from the obtained information, we create a continuous line with the area of large brightness change as the boundary (however, it may have a certain width). This method obtains binarized image information that can be recognized as 2D information, and since the processing involves arithmetic processing of two-dimensional information, there is a disadvantage that as the amount of processing information increases, the processing time becomes exponentially longer.

Incidentally, if the processing time required for fruit recognition becomes long, this will lead to disadvantages, such as making it difficult to efficiently perform work on fruits such as fruit harvesting when used to control fruit working machines, for example. Therefore, it is desired to be able to perform fruit recognition as quickly as possible.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to extract binarized image information as areas in which a plurality of fruits that are crowded or overlap each other are separated one by one. The point is that the processing can be performed at high speed with a simple configuration.

[Means for solving problems]

The fruit recognition device according to the present invention is characterized by controlling the opening and closing of the binary color image information as an opening/closing control signal, thereby masking the grayscale image information with the fruit color to be recognized. A gate means is provided for determining a region corresponding to the fruit color and extracting grayscale image information of the region, and the grayscale image information extracted by the gate means is input to the edge image detecting means. Its functions and effects are as follows.

[Effect]

That is, the grayscale image information of the captured image information is determined and extracted from the color image information of the captured image information, and the area corresponding to the color of the recognition target fruit is extracted and binarized. By masking, the grayscale image information in the same area as the binarized color image information is extracted, and the grayscale image information from which the grayscale image information in areas other than the recognition target fruit color has been removed is input to the edge image detection means. Let it happen.

〔Effect of the invention〕

Therefore, the grayscale image information masked with the binary color image information does not include unnecessary information other than the fruit to be recognized, and the amount of information processed by the edge image detection means can be significantly reduced. In other words, high-speed processing is possible because the amount of information to be processed is small.

In addition, since image information other than the recognition target fruit is removed in advance before image processing, unnecessary noise components are extremely small in the binarization process of the grayscale image information, and the binarized image information The reliability of the separation and extraction results has also improved significantly.

Furthermore, the gradation image information in which the gradation image information of the captured image information is masked with the binarized color image information and the 2
Valued color image information can be obtained through real-time processing, so
The time required for this mask processing does not affect the processing time required for fruit recognition.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

FIG. 1 is a block diagram of a fruit recognition device that obtains binarized image information in which each fruit to be recognized is separated from image information taken outdoors. It is configured as a device for recognizing warm-colored fruits, and by subtracting the color signal B of the blue component from the color signal R of the red component that makes up the color image, it is possible to recognize branches, leaves, sky, etc. other than the fruit color. The color component corresponding to the background object is removed, and the set recognition target fruit color C is further removed.
By masking with the binary color image information F1 extracted by determining the area corresponding only to the fruit color C to be recognized, the area corresponding only to the recognition target fruit color C is determined and obtained. Grayscale image information S ₁
From the binarized edge image information F ₂ that is binarized,
It is configured to obtain binarized image information _F3 in which each of the fruits that appear to overlap is separated.

That is, image information captured by the color video camera 1 as an imaging means (see FIG. 8A)
NTSC format color video signal output as
S ₀ is a vertical synchronization signal by NTSC decoder 2.
VD, horizontal synchronization signal HD, each color signal, red signal R, blue signal B, and luminance signal Y as grayscale image information. The difference (R-B) between the red signal R and the blue signal B is calculated and compared with a preset threshold value Cref that is preset based on the set recognition target fruit color C, the brightness of the imaging state, etc.
Binarized color image information F ₁ which is the binarized image information of the color corresponding only to the recognition target fruit color C (see Fig. 8B)
is converted to

As shown in FIG. 5, the luminance signal Y uses the binarized color image information _F1 as a control signal,
By passing the gate circuit 4 which is open only while the binary color image information F ₁ is at the "H" level, that is, only in the section corresponding to the fruit color C, the binary color image information F ₁ is used as a mask to allow the recognition described above. Target fruit color C
Extracted as grayscale image information _S1 corresponding to only
The A/D converter 5 converts the image into digital grayscale image information _S2 (see FIG. 8C) consisting of 128 x 128 pixels/pixel with a resolution of 8 bits/pixel, and the data is sent via the data buffer 6 to the first image. The image is temporarily stored in the image memory 7.

The process of obtaining the binarized color image information F ₁ and the process of A/D converting the grayscale image information S ₁ and storing it in the first image memory 7 as digital grayscale image information S ₂ for one screen are as follows: The processing operation is controlled by the timing circuit 8 so that it is performed simultaneously and in parallel in synchronization with each change timing of the vertical synchronization signal VD and the horizontal synchronization signal HD.
Further, a strobe control circuit 9 activated by a control signal from the timing circuit 8 controls the light emission timing and the light emission so that the strobe device 10 serving as the illumination means emits light in synchronization with the imaging operation by the camera 1. It is configured to control the intensity.

Further, the digital grayscale image information _S2 stored in the first image memory 7 is converted into a composite video signal by the CRT controller 11 and the TV signal generator 12, and then sent to the monitor television 13.
It is displayed as follows.

The digital grayscale image information S ₂ is then sent to the control processor CPU ₁ and the numerical calculation processor.
The CPU ₂ performs arithmetic processing such as noise removal and contour extraction, and then converts it into binary edge image information F ₂ (see Figure 8 D), which is then stored in the second image memory. 14 is temporarily stored.

Next, grayscale image information S ₂ within the range corresponding to the recognition target fruit color C stored in the first image memory 7
Based on the gradation value of , those whose values are positive are restored as the binarized color image information F ₁ , and the 2 stored in the second image memory 14 is restored from the binarized color image information F ₁ . By subtracting the digitized edge image information F ₂ , calculate the binarized image information F ₃ (see FIG. 8 E) in which each fruit is separated, and store it in the second image memory 14. . Therefore, the grayscale image information _S2 masked with the binary color image information _F1 has the same image information as the binary color image information _F1 in the portion where the brightness information is not zero. , without separately storing the binarized color image information _F1 .
Since the binary color image information _F1 can be easily restored from the masked grayscale image information _S2 , there is no need to provide an image memory for storing the binary color image information _F1 itself.

Then, based on the binarized image information _F3 in which each fruit has been separated, the number of recognized fruits, their coordinate positions, etc. are calculated, and the information is sent to the host computer via the interface device 15.
The recognition result is transmitted to CPU ₀ , and the overall operation is controlled. In addition, in Figure 1, 1
Reference numeral 6 denotes a memory for temporarily storing operating programs and calculation data for each of the processors CPU ₁ and CPU ₂ , and for storing various data exchanged with the host computer CPU ₀ .

The configuration of each part will be explained in detail below.

As shown in FIG. 4, the color separation circuit 3 is configured so that its operation is controlled by a timing controller 30 so as to operate in synchronization with the horizontal synchronization signal HD and the vertical synchronization signal VD. . The difference R between the red component color signal R and the blue component color signal B output from the decoder 2 is
-B is calculated by a differential amplifier 31, its output signal is amplified to a predetermined level by a video amplifier 32 whose amplification degree is set by a gain setting resistor R ₁ , and a comparator 33 calculates the amplification degree set by a resistor R ₂ . It is compared with a threshold value Cref and converted into TTL level binary image information _F1 .
By the way, the threshold value Cref is set in advance based on the set recognition target fruit color C, the brightness of the imaging state, and the like.

As shown in FIG. 6, the strobe control circuit 9 includes two one-shot multivibrators 90 and 91 activated by a trigger signal VREQ corresponding to the start position of the blanking period of the vertical synchronization signal VD. be.

Then, as shown in FIG. 7, after a predetermined time _t1 delayed by the first multivibrator 90 has elapsed, the next multivibrator 91 is activated, and its output time _t2 corresponds to the light emission intensity of the strobe device. Strobe device 10 such that 10 emits light.
It is configured to turn on a photocoupler 92 that transmits a light emission signal to. In other words, the strobe device 10 automatically emits light at a predetermined illuminance every time in synchronization with the imaging of one screen by the camera 1 and during the retrace period of the vertical synchronization signal VD.

The overall operation will be described below based on the flowchart shown in FIG.

That is, when the power is turned on, the control processor CPU ₁ initializes the contents and calculation data of each memory 7, 14, 16, and transfers the data to the host computer.
Self-holds while waiting to receive a start signal from CPU ₀ . The host computer CPU ₀ confirms that the control processor CPU ₁ is self-holding, sets the image processing start flag, notifies the control processor CPU ₁ of the start of processing, and then controls the control processor CPU ₁ . It enters a waiting state for processing to end.

When the start flag is set, the control processor CPU ₁ starts imaging processing by the camera 1, takes in image information _S0 for one field, and converts it into binary color image information of the color difference signal RB.
Load the grayscale image information S ₂ masked with F ₁ and 2
It performs processing such as labeling processing for converting into values and calculation of the center of gravity position of the recognition target, and then enters a standby state.

On the other hand, the host computer CPU ₀ confirms that the control processor CPU ₁ is in a standby state, reads out the calculation result, resets the start flag, and restarts the control processor CPU ₁ . . The above operation is then repeated as many times as necessary.

Next, based on the flowchart shown in FIG. 2, the digital grayscale image information _S2 is binarized,
The operation for obtaining the binarized image information _F3 corresponding only to the recognition target fruit color C in a state in which each of the plurality of fruits is separated will be explained.

That is, when the start flag is set, the binary signal output from the color separation circuit 3
Gate circuit 4 and A/D opened and closed by F ₁
Luminance signal Y for one field via converter 5
is input to the first image memory 7 as digital gradation image information _S2 masked with the fruit color C to be recognized.

Then, using the masks shown in Figure 9 A and B,
Based on the "Sobel Operator" method, the brightness changes of the central pixels j and i for three pixels on the top, bottom, left, and right are first differentiated and binarized. After that, erase the area with a small area (set the value to “0”), and the remaining “1”
The area is thinned, and binarized edge image information F ₂ of the grayscale image is extracted and stored in the second image memory 14 .

The binarized edge image information _F2 is converted into binarized edge image information connected by continuous lines by performing line thinning processing. Note that this thinning process, for example, as shown in FIG. This is done by calculating the relationship between In other words, as shown in FIG. 10B, the number of connections Nc at the pixel of interest _X0 whose value is "1" is the value f of the eight adjacent pixels _X2 to _X8 .
It is represented by the following formula using (X ₁ ) to f(X ₈ ).

Nc(X ₀ )= 〓 ^k=j (f(Xk)−f(Xk)・f(Xk _-1 )・f(Xk ₊₂ )) However, j={1, 3, 5, 7,}, Let X ₉ = X ₁ .

Therefore, Nc(X ₀ ) takes values from 0 to 4, but 1
X ₀ becomes erasable when . (Connectivity will be maintained even if erased.) However, if it represents the end of a thin line (line with a line width of 1) as shown in Figure 10 C, D, E, F, please do not erase it. do. Then, by sequentially erasing the erasable parts of each pixel, it is possible to make the line thinner.

Then, by subtracting this thinned binary edge image information F ₂ from the binary color image information F ₁ , the separated binary values that separate each of the clustered fruits are separated. Convert image information to _F3 .

Next, using the mask shown in _FIG . Processing is performed to convert into binarized image information in which multiple fruits are completely separated. In other words, if any of the pixels adjacent to the pixel of interest j, i on the top, bottom, left, or right is "0", the pixel of interest j, i is erased as "0", and all three pixels on the top, bottom, left, and right are erased. If it is “1”, the pixel of interest j,
Processing is performed by setting i to "1".

Thereafter, by performing so-called labeling processing on the binarized image information _F3 , the number and size of the recognized fruits are calculated, the center of gravity position of each recognized fruit is calculated, and the results are sent to the host It is transmitted to computer CPU ₀ .

[Another example]

In the embodiment described above, the binary color image information F ₁ itself is not stored, but is restored as necessary from the gray scale image information S ₂ masked with this binary color image information F ₁ . , an image memory for separately storing the binarized color image information _F1 may be provided. and,
In this case, the image information stored in the first image memory 7 that stores the grayscale image information _S2 is sequentially processed,
It may also be converted into separate binary image information _F3 . In either case, the required image memory is smaller than the case where both the binary color image information F ₁ and the gray scale image information S ₂ are stored in addition to the image memory for calculation.

[Brief explanation of drawings]

The drawings show an embodiment of the fruit recognition device according to the present invention, FIG. 1 is a block diagram showing the overall configuration of the fruit recognition device, FIG. 2 is a flowchart showing the operation of the fruit recognition device, and FIG. 3 is a fruit recognition device. A flowchart showing the relationship between the device's control processor and the host computer, Figure 4 is a block diagram showing the configuration of the color separation circuit, Figure 5 is a time chart showing the operation of the gate circuit, and Figure 6 is a diagram of the strobe control circuit. FIG. 7 is a circuit diagram showing the configuration, FIG. 7 is a time chart showing its operation, FIG. 8 A to E are explanatory diagrams of image information, and FIG.
FIGS. 10A to 10F are explanatory diagrams of the thinning process. 1... Imaging means, 2... Color binarization means, 4...
Gate means, C...fruit color to be recognized, _S0 ...captured image information, Y...shade information of the captured image, _S1 ...shade image information of fruit color range to be recognized, _F1 ...fruit color to be recognized F ₂ ...binarized edge image information of a grayscale image, F ₃ ...separated binary image information.

Claims (1)

[Claims] 1 Determine the area corresponding to the recognition target fruit color C set from the color image information in the image information S0 captured by the image capturing means 1, and convert it into the binarized color image information F.
Edge image detection means 3 for extracting the color as 1, and edge image detection for determining a portion of the captured image information S0 that has a larger brightness change than the grayscale image information Y, and extracting it as the binary edge image information F2. and a subtraction means for subtracting the binarized edge image information F2 from the binarized color image information F1 to separate and extract a region corresponding to the recognition target fruit from the captured image information S0. The fruit recognition device obtains digitized image information F3, and controls the opening and closing of the binarized color image information F1 as an opening/closing control signal, thereby converting the gradation image information Y into a fruit color C.
A gate means 4 is provided for masking and extracting the gray image information S1 of the area corresponding to the fruit color C from the gray image information Y, and the gray image information S1 extracted by the gate means 4 is A fruit recognition device configured to receive input using edge image detection means.