JPH08154450A - Fruit recognizing device - Google Patents

Abstract

PURPOSE: To provide a method for reducing the effect of irregular brightness especially on a green fruit as far as possible when forming a binary value image in a pre-stage of fruit recognition. CONSTITUTION: An automatic fruit harvester 1 automatically harvests fruits by automatically operating a fruit harvesting manipulator by a visual information of a color CCD camera 3 and has a brightness reference-calculating means for calculating the brightness reference of the color image: (R+G+B)/3 by adding the brightness of R, G, B signals of a color CCD camera 3, an image binary value-making means by calculating brightness difference between G-image based on G-signal of the color CCD camera 3 and the brightness reference and making the image binary value, a fruit recognition means by analyzing binary image obtained by the image binary value-making means and recognizing a fruit, and a manipulator controlling means for operating the fruit harvesting manipulator 6 based on a fruit recognizing result of the fruit recognizing means and outputting the operation signal. A fruit, especially a green fruit is most clearly recognized by using the binary-valued image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic fruit harvester for automatically harvesting fruits by automatically operating a fruit harvesting manipulator based on visual information of a color CCD camera, and more particularly to image processing of a fruit recognition device thereof.

[0002]

2. Description of the Related Art In a conventional fruit recognition device, a detected image signal obtained by photographing with a camera is binarized based on a certain threshold value, and fruit recognition is performed based on the binarized image. Was there.

[0003]

In the conventional fruit recognition device, when there is a leaf behind a fruit such as cucumber and the fruit and the leaf appear to overlap with each other, the contour of the fruit cannot be accurately detected due to the uneven brightness. There was a problem.

In order to solve such a problem, the present invention has been made in order to reduce the influence of unevenness in lightness as much as possible on a greenish fruit in preparing a binarized image in the preceding stage of fruit recognition. .

[0005]

In order to achieve the above object, the present invention has the following constitution.

That is, the present invention relates to an automatic fruit harvester for automatically harvesting fruits by automatically operating the fruit harvesting manipulator based on the visual information of the color CCD camera, in which the R, G, B signals of the color CCD camera are added to the luminance. Then, the brightness reference calculating means for calculating the brightness reference (R + G + B) / 3 of the color image, and the brightness difference between the G image based on the G signal of the color CCD camera and the brightness reference are obtained to obtain an image 2
Image binarization means for binarizing, fruit recognition means for recognizing fruits by analyzing the binarized image by the image binarization means,
And a manipulator control unit that outputs an operation signal for operating the fruit harvesting manipulator based on the fruit recognition result of the fruit recognition unit.

[0007]

The lightness reference calculating means is R of the color CCD camera,
Luminance reference of the color image (R +
G + B) / 3 is calculated. The image binarizing means binarizes the image by obtaining a difference in brightness between the G image based on the G signal of the color CCD camera and the brightness standard. The fruit recognition means analyzes the binarized image obtained by the image binarization means and recognizes fruits. The manipulator control means outputs an operation signal for operating the fruit harvesting manipulator based on the fruit recognition result of the fruit recognition means.

[0008]

Embodiments of the present invention will be described below with reference to the drawings.

1 and 2 are views showing an automatic fruit harvester 1, FIG. 1 is a side view of the machine, and FIG. 2 is a rear view of the machine as seen from the rear. The automatic fruit harvester 1 includes a traveling unit 2 and
The color CCD camera 3 mounted on the traveling unit 2, the illumination unit 4, the visual unit 5, the manipulator 6, and the tilt frame 7
And a carry section 8 and a guide wire detection sensor 9. Color CCD camera 3, lighting unit 4, and visual unit 5
Then, the manipulator 6 is integrally mounted on a base that is mounted to be vertically movable along the guide rails of the inclined frame 7.
The tilt frame 7 has a structure in which the tilt angle can be arbitrarily adjusted, and is tilted so as to be substantially parallel to the tilt of the cultivation rack of the cucumbers 10 facing each other.

As shown in FIG. 3, the visual part 5 comprises a position sensor 21, an illuminance sensor 22 and a control part 23. The color CCD camera 3, the position sensor 21, and the illuminance sensor 22 are connected to the input side of the control unit 23.
The manipulator 6 is connected to the output side of the. Control unit 2
3 includes an input circuit 24, a CPU 25, and an output circuit 26.

The manipulator 6 has a structure as shown in FIG. 4, and comprises a motor 31, hand bases 32 and 33, a screw rod 34, a pantograph 35, and a grip 36. The screw rod 34 has different screw directions in the upper half portion and the lower half portion. The hand base portion 32 is screwed into the upper half portion, and the hand base portion 33 is screwed into the lower half portion, and the motor 31 is connected to one end thereof. One end of a pantograph 35 is attached to the hand bases 32 and 33, and a grip 36 is attached to the other end of the pantograph 35.

As shown in FIG. 5, the guide wire detection sensor 9 is provided with a pair of left and right sensors, one set each in front of and behind the upper part of the body of the automatic fruit harvesting machine 1. The pair of left and right guiding line detection sensors 9 are arranged symmetrically with respect to the center line of the automatic fruit harvesting machine 1. The guide wire 11 is installed on the ceiling of the house.

The automatic fruit harvester 1 automatically travels inside the house by the electric traveling unit 2. Color CCD camera 3
Takes a greenish fruit such as cucumber 10.
The lighting unit 4 uses a color CC when photographing the fruit to be harvested.
Light is emitted from four directions of the D camera 3. The position sensor 21 of the visual part 5 scans the visual field of the color CCD camera 3 horizontally and vertically to measure the distance to the cucumber 10 of the object. The illuminance sensor 22 of the visual part 5 is a color CC.
The illuminance at the time of capturing an image taken by the D camera 3 is detected. Various detection signals detected by the color CCD camera 3, the position sensor 21, and the illuminance sensor 22 are input to the CPU 25 via the input circuit 24 of the control unit 23. CPU2
5 analyzes the input various detection signals to recognize fruits and outputs various operation signals for operating the manipulator 6 via the output circuit 26.

Fruit recognition is performed by analyzing a binarized image obtained by binarizing a color image taken by the color CCD camera 3. This binarization processing will be described with reference to the flowchart shown in FIG. When the process starts (step 10
1) First, a color image photographed by the color CCD camera 3 is input pixel by pixel (step 102). Next, the brightness difference G- (R + G + B) / 3 between the G image and the lightness reference is calculated for the input color pixel (step 10).
3). The brightness difference calculated here is compared with the magnitude of a predetermined threshold value (step 104). If the brightness difference is greater than or equal to the threshold value, the value of the pixel is set to 1 (step 105). When the brightness difference is smaller than the threshold value, the value of the pixel is set to 0 (step 106).
After that, it is determined whether or not all the pixels have been input (step 107). When all the pixels have been input, the processing is ended (step 108), and otherwise the processing returns to step 102.

In order to recognize whether or not the pixel group is a fruit by analyzing the binarized image, the ratio of the maximum lengths in the vertical and horizontal directions of each pixel group is not less than a predetermined value, or It is determined by determining whether a predetermined number or more of lines having a width in the horizontal direction within a certain range continuously exist in a part of each pixel group (see, for example, Japanese Patent Publication No. 5-0231).

When the screw rod 34 is normally or reversely rotated by using the motor 31 provided in the manipulator 6, the hand bases 32 and 33 screwed into the screw rod are moved toward and away from each other. The pantograph 35 expands and contracts when the hand bases 32 and 33 approach and separate, and the grip portion 26 at the tip moves forward and backward as it expands and contracts. Utilizing this expansion and contraction movement of the pantograph 35, the tip of the manipulator 6 is approached to the fruit to be harvested. A fruit harvesting device is attached to the grip portion 36, the cucumber 10 to be harvested is sandwiched, the fruit pattern is cut, and the harvested cucumber is put into the carry portion 8.

By rotating the screw rod 34 at a constant speed, the hand base 3
When the approach speeds of 2 and 33 are made constant, the extension amount of the manipulator 6 with respect to time draws a non-linear curve as shown in FIG. The expansion amount of the manipulator 6 with respect to this time can be linearized by logarithmizing it. There is a method of using a square root as a non-linear approximation function, but the logarithm is optimal in this mechanism.

Conventionally, the expansion amount was calculated by a method such as using a data table or dividing the time. However, it is necessary to use an approximation function obtained by logarithmizing the expansion amount of the manipulator 6 with respect to this time and performing linear conversion. This makes it possible to easily calculate the extension amount with high accuracy. In addition, the tip of the manipulator 6 can be more accurately approached to the target fruit in combination with the characteristics of the pantograph mechanism that the movement starts early and ends late.

The guide wire detection sensor 9 detects the magnetism from the guide wire 11. The vertical distance from the guide wire 11 is 20c
When the distance between the left and right guide wire detection sensors 9 is 30 cm, the output voltage due to magnetic induction with respect to the lateral displacement of the sensors changes as shown in FIG. Therefore, when the automatic fruit harvesting machine 1 is displaced so that the output voltages of the left and right guide wire detection sensors 9 have the same value, the midpoint of the line connecting the left and right guide wire detection sensors 9 is directly below the guide wire 11. . Even when only one set of the midpoints of the lines connecting the left and right guide wire detection sensors 9 is directly below the guide wire 11, the guide wire detection sensors 9 of the other set are connected to each other.
When it is not located right below the center line of the automatic fruit harvesting machine 1 as shown in FIG. The direction of the center line of the automatic fruit harvesting machine 1 and the direction of the guide wire 11 are completely aligned only when the output voltages of the left and right guide wire detection sensors 9 have the same value before and after simultaneously. In this way, by controlling the traveling of the automatic fruit harvesting machine 1 so that the center line of the automatic fruit harvesting machine 1 and the direction of the guide line 11 coincide with each other, the automatic fruit harvesting machine 1 is automatically run along the guiding line 11. be able to.

Conventionally, the guide wire 11 was attached to the ground. However, it is troublesome to remove the guide wire 11 at the end of harvesting and when disinfecting soil, or to attach the guide wire 11 again at the next harvesting work. It was hanging. In the embodiment shown in the drawings, since the guide wire 11 is erected on the house ceiling, the guide wire 11 once erected can be continuously used thereafter. Therefore, it is troublesome to remove the guide wire 11 or attach it again. It can be omitted.

[0021]

According to the present invention, the color C having the above-mentioned constitution is used.
The brightness standard (R + G + B) / 3 of the color image is calculated by adding the luminances of the R, G, and B signals of the CD camera to obtain the color C
The G image based on the G signal of the CD camera and the brightness difference on the basis of the brightness are obtained to binarize the image. Therefore, according to the present invention, from various experimental results, it is possible to reduce the influence of unevenness in lightness, particularly for green fruits, and it is possible to recognize green fruits most clearly.

[Brief description of drawings]

FIG. 1 is a side view of an automatic fruit harvester 1 according to an embodiment of the present invention.

FIG. 2 is a rear view of the automatic fruit harvester 1 according to the embodiment of the present invention.

FIG. 3 is a block diagram of a visual part 5 according to an embodiment of the present invention.

FIG. 4 is a structural diagram of a manipulator 6 according to an embodiment of the present invention.

FIG. 5 is a diagram showing the position of the guide wire detection sensor 9 according to the embodiment of the present invention, and showing a state in which the guide wire 11 and the direction of the automatic fruit harvester 1 match.

FIG. 6 is a view showing a state in which the guide wire 11 and the automatic fruit harvester 1 in FIG. 5 are inclined.

FIG. 7 is a flowchart illustrating a binarization process according to the embodiment of this invention.

FIG. 8 is a diagram showing the relationship between the expansion amount and the time of the manipulator 6 according to the embodiment of this invention.

FIG. 9 is a relationship diagram of the output voltage with respect to the displacement of the guide wire detection sensor 9 according to the embodiment of the present invention.

[Explanation of symbols]

 1 Automatic Fruit Harvester 2 Traveling Section 3 Color CCD Camera 4 Illumination Section 5 Visual Section 6 Manipulator 7 Tilt Frame 8 Carry Section 9 Guidance Wire Detection Sensor 10 Cucumber 11 Guidance Wire 21 Position Sensor 22 Illuminance Sensor 23 Control Section 24 Input Circuit 25 CPU 26 Output Circuit 31 Motors 32, 33 Hand Base 34 Screw Rod 35 Pantograph 36 Gripping Part

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location G06T 5/00

Claims (1)

[Claims] 1. An automatic fruit harvester for automatically harvesting fruits by automatically manipulating a fruit harvesting manipulator based on visual information of a color CCD camera, wherein the R, G, B signals of the color CCD camera are added to luminance to produce a color. Lightness reference calculation means for calculating the lightness reference (R + G + B) / 3 of the image, and image binarization means for binarizing the image by obtaining the brightness difference between the G image based on the G signal of the color CCD camera and the lightness reference. A fruit recognition means for recognizing a fruit by analyzing the binarized image by the image binarization means, and a manipulator control for outputting an operation signal for operating the fruit harvesting manipulator based on the fruit recognition result of the fruit recognition means. A fruit recognition device comprising: