JP3139074B2 - Visual devices such as fruit harvesting robots - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual search device provided in a fruit harvesting robot for searching for a crop.

[0002]

2. Description of the Related Art While self-propelled along planting crops,
Fruit harvesting robots that automatically harvest mature fruits have been developed. A visual device for searching for a crop provided in this kind of fruit harvesting robot is provided with an image sensor camera having a filter that passes only light of a hue component unique to the fruit, and binarizes an image captured by the image sensor camera. In many cases, harvestable fruits are detected.

[0003]

However, when the cultivated crop is cucumber, the hue of the fruit and the leaf are both green, so that the image taken by the image sensor camera that picks up only the light of the green hue component contains the fruit. And both leaves are detected. When the fruits and leaves are located independently, the two can be distinguished from each other based on the difference in shape and size, but when they overlap, the outer shape of the image becomes complicated, and it is not possible to recognize only the fruits. It was difficult. Therefore, the present invention, for example, the same hue components of fruits and leaves like cucumbers,
In addition, even when the fruit and the leaf overlap one another, it is an object of the present invention to be able to separate and identify each of them.

[0004]

In order to solve the above problems, the present invention has the following arrangement. That is, the visual device such as a fruit harvesting robot according to the present invention includes: a camera that captures only the light of the image of a particular color component, horizontal direction of the object
Scan in the vertical and vertical directions to determine the distance to the object.
A distance measuring element which is imaged by the camera.
Binarized image based on the threshold value
The contour of the image is obtained, and the 2
Determine whether the valued image is that of a fruit, and
The center point of the binary image only if it is expected to be
Then, the periphery of the center point is scanned with the measuring element to measure
The shape of the fruit is calculated from the scanning points whose distance is within the specified range.
It is characterized in that it is configured to

[0005]

First, the input signal of the camera is binarized to obtain an image in which a plurality of objects, for example, fruits and leaves overlap. Next, the distance measuring element is used to measure the horizontal and vertical directions (two orthogonal directions).
(Axial direction) to measure the distance to the object, and analyze the image based on the measurement result. A plurality of objects are separated and identified based on the distribution of distance values.

[0006]

FIG. 3 shows the use state of a cucumber fruit-harvesting robot. The fruit-harvesting robot 1 includes an electric traveling cart 2 as a moving means, and an inclined frame 4 having a guide rail 3 is installed on the traveling cart, on which a fruit-harvesting manipulator 6 and a visual device 7 move up and down. It is provided freely. The inclined frame 4 is pivotally supported by the traveling trolley 2 by a hinge 8, and the back side is supported by a support link 9. The lower end of the support link 9 is fixed to an appropriate position of the elongated hole 10, and by changing the fixing position, the inclined frame 4 is fixed.
Can be arbitrarily adjusted.

The manipulator 6 is provided on the guide rail 3.
A base 12 which can be raised and lowered along the base, a main body 13 provided on the base so as to be rotatable in a horizontal plane, an articulated arm 14 provided on the main body, and a distal end of the articulated arm. And a fruiting hand 15 provided. The articulated arm 14 connects the upper arm 17 on the main body side and the forearm 18 on the thinning hand side to the elbow 1
9, the arm can be turned up and down, bent and stretched, and the fruit picking hand 15 can be turned up and down in the same manner as a human arm. When the articulated arm 14 is bent, the elbow portion 19 is bent so as to come down. The fruit picking hand 15 is configured to grasp the fruit and cut the fruit handle.

As shown in the block diagram of FIG. 1, the visual device 7 includes an image sensor camera (CCD) 31 for capturing only a green hue component and a distance measuring element (PSD) 32 for measuring a distance to an object. And a CPU 33 for analyzing and processing input signals from the visual parts 7a. The visual section 7 a is provided at the upper end of the manipulator main body 13, and the CPU 33 is built in the main body 13.

The distance measuring element 32 has the structure shown in FIG. The light emitted from the light projector 40 is reflected by the reflecting mirror 41 to irradiate the object 42, and the reflected light is reflected by the reflecting mirror 4.
The light is reflected at 3 and received by the light receiver 44. Reflector 41,
Since the distance of 43 is constant, the distance to the object 42 is measured based on the principle of triangulation. Reflector 41, 4
3 is changed to the stepping motors M ₁ and M ₂ , horizontal scanning in the figure is performed, and the entire case 46 is rotated by the stepping motor M ₃ in the figure.
Performing a vertical scanning that.

Before starting work, the visual device 7 is adjusted by the following method (see FIG. 4). First, as shown in FIG. 5, a plate 48 on which a prescribed pattern 47 is marked is provided in front of the camera 31, and this is imaged by the camera 31. Next, the distance value d ( 4 ) of the four corners (Q ₁ , Q ₂ , Q ₃ , Q ₄ ) of the specified pattern 47 by the distance measuring element 32.
Number) was measured, to drive the stepping motor M _1, M _2, M ₃ such that they become equal. As a result, the image input range of the camera 31 and the measurement range of the distance measuring element 32 match. The scan angle at this time is stored.

The processing for searching for fruits is performed in the order shown in the flowchart of FIG. In the illustrated example, the resolution of the input signal level is 64. The camera 31 is set so that the input signal level becomes smaller than 64 in all pixels.
Is adjusted, and a threshold value is calculated from the average value of the input signal levels.
Value. Subsequently, noise in the image is removed. It is determined whether the number of “1” pixels after noise removal is equal to or greater than a predetermined value P. If the number of pixels is equal to or greater than the predetermined value P, the process proceeds assuming that the camera 31 has captured something. The visual section 7a is moved assuming that nothing is reflected on the camera 31.
When the number of “1” pixels is equal to or greater than a predetermined value P, the outline of the binarized image is obtained, and it is further determined whether or not the object of the image is a fruit based on the ratio of vertical and horizontal pixels. Scanning with the distance measuring element 32 only when If it is determined that it is not a fruit, the visual part 7a is moved. Note that immature fruits are removed as noise because the light reflection area is small. As described above, since the scanning is performed by the distance measuring element 32 only when there is a fruit in the image of the camera 31, the processing time for fruit recognition is reduced.

When the camera 31 captures the fruit, the distance measurement element 3 is input before the image signal is input to the CPU 33.
In step 2, the center of the screen is horizontally scanned to determine the shortest distance, and the visual part 7a is moved back and forth so that the shortest distance is about 30 cm (see FIG. 7). Since the distance from the camera 31 to the object is always substantially the same, it is possible to obtain information under the same condition where the influence of illumination is constant. Also, since the size of the fruit can be roughly determined only by the number of fruit recognition pixels,
Processing time is reduced.

To determine the size of the fruit, the processing shown in the flowchart of FIG. 8 is performed. First, the image of the image sensor camera 31 is binarized into a fruit portion and a background portion, and a center point C of the fruit portion is obtained (see FIG. 9). Then, the vicinity of the center point C is horizontally and vertically scanned by the distance measuring element 32, and when the distance value of the center point is dc, the distance value is dc.
By extracting the scanning points ± α, the shape of the fruit is determined. This allows only ripe fruits to be harvested.

It should be noted that the length and width of an image which is considered to be a fruit is calculated based on the pixel pitch, and is compared with a prepared comparison table to determine whether or not the image is actually a fruit (see FIG. 10). ). For example, a typical length-width relationship for mature cucumbers is as shown in FIG. As described above, by confirming that the fruit is obtained by utilizing the relationship between the length and the width (diameter), it is possible to prevent the stem from being erroneously damaged during harvesting.

When a plurality of objects are input in one screen, a histogram of distance value distribution of each image (see FIG. 13)
And leaves and fruits are identified based on the frequency and area (see FIG. 12).

Further, for example, as shown in FIG.
And the fruit (cucumber) 51 overlaps front and back,
Both are projected simultaneously on one screen. In such a case,
The analysis shown in the flowchart of FIG. 14 is performed to separate them. First, an object is photographed by the image sensor camera 31, and is binarized into a green hue component and other hue components. Since both the fruit and the leaf are green, the binarized image has a shape in which both are combined as shown in FIG. Next, the contour line of the binarized image is obtained, and the distance measuring element 32
Scan horizontally and vertically. Then, a difference between the distance values on the contour line is obtained, and as shown in FIG. 17, a line segment A having a distance value smaller than the threshold value L and a distance value larger than the threshold value L based on the threshold value L, as shown in FIG. The outline is separated into a line segment B. Line A represents the outline of the leaf, and line B represents the outline of the cucumber.

In the above analysis procedure, instead of taking the difference between the distance values on the contour line, as shown in FIG. 19, a histogram of the distance values within the contour line is created, and the contour line of the leaf 50 and the fruit 51 is obtained from the histogram. May be separated (see FIG. 18).

When the visual device 7 finds a harvestable fruit, the manipulator 6 performs a predetermined operation to harvest the fruit. When all the fruits within the visual field of the visual section 7a have been harvested, the direction of the visual section 7a is changed, and the harvest of the fruits is repeated.
When all the fruits within the entire working range of the manipulator 6 have been harvested, the carriage 2 is moved to move to the next plant.

[0019]

As described above, the visual device of the fruit-harvesting robot according to the present invention analyzes the image obtained by the camera based on the measurement result of the distance measuring element, so that the leaves and the fruits can be extracted like cucumbers. Even if the crops are the same color,
Both can be reliably separated and identified.

[Brief description of the drawings]

FIG. 1 is a block diagram of a visual device.

FIG. 2 is a diagram showing a configuration of a distance measuring element.

FIG. 3 is a diagram illustrating a use state of the fruit harvesting robot.

FIG. 4 is a flowchart of a first visual device control.

FIG. 5 is a diagram illustrating the concept of the first visual device control.

FIG. 6 is a flowchart of a second visual device control.

FIG. 7 is a flowchart of a third visual device control.

FIG. 8 is a flowchart of a fourth visual device control.

FIG. 9 is a diagram of an image for supplementing the description of the fourth visual device control.

FIG. 10 is a flowchart of a fifth visual device control.

FIG. 11 shows a standard length-diameter relationship for mature cucumbers.

FIG. 12 is a flowchart of a sixth visual device control.

FIG. 13 is a histogram of distance values to supplement the description of the sixth visual device control.

FIG. 14 is a flowchart of seventh visual device control.

FIG. 15 is a diagram illustrating a concept of seventh visual device control.

FIG. 16 is a diagram of an image to supplement the description of the seventh visual device control.

FIG. 17 is a diagram illustrating a relationship between a contour line and a difference for supplementing the description of the seventh visual device control.

FIG. 18 is a flowchart of eighth visual device control.

FIG. 19 is a histogram of distance values to supplement the explanation of the eighth visual device control.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Fruit harvesting robot 6 Manipulator 7 Visual device 15 Fruit picking hand 31 Image sensor camera 32 Distance measuring element 33 CPU 50 Leaves 51 Fruit

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-60-15780 (JP, A) JP-A-60-39279 (JP, A) JP-A-60-116077 (JP, A) JP-A-2- 234013 (JP, A) JP-A-63-285410 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00 A01D 46/24-46/253 B25J 19/04 G05D 1/02-1/03

Claims (1)

(57) [Claims] 1. A camera that captures only an image of light of a specific hue component, and a camera that scans an object in horizontal and vertical directions.
A distance measuring element for measuring the distance to the object;
The image taken by the camera is based on a threshold.
To obtain a contour line of the binarized image.
The binarized image is that of a fruit based on the aspect ratio of the contour line
Judge whether the fruit is expected
Determining the center point of the binary image and measuring the periphery of the center point
Scans with the element and scans the measured distance within a predetermined range.
The feature is that the shape of the fruit is determined from the inspection points.
A visual device such as a fruit harvesting robot.