JPH0554124A - Visual device for fruit harvesting robot or the like - Google Patents

Abstract

PURPOSE:To respectively separately identify fruits like cucumbers, for example, even when the fruits and the leaves have the same hue component and are overlapped back and forth with respect to the visual device for a fruit harvesting robot. CONSTITUTION:This device is equipped with a camera 31 for picking up only the image of light having a specified hue component, distance measuring element 32 for measuring a distance to an object and CPU 33 for analyzing and processing the image of the camera based on the measured result of the distance measuring element 32.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual device for searching a harvested product provided in a fruit harvesting robot.

[0002]

[Prior Art] While self-propelled along the planting line of cultivated crops,
A fruit harvesting robot has been developed that automatically harvests mature fruits. A visual device for searching a harvested product provided in a fruit harvesting robot of this type includes an image sensor camera having a filter that passes only light of a hue component peculiar to fruit, and binarizes an image captured by the image sensor camera. By doing so, many of them are designed to detect harvestable fruits.

[0003]

However, when the cultivated crop is cucumber, the hues of both the fruit and the leaves are green, so the image taken by the image sensor camera that captures only the light of the green hue component is the fruit. And leaves are both detected. When fruits and leaves are located independently, they can be distinguished from each other based on the difference in shape and size, but when they overlap, the outline of the image becomes complicated and it is not possible to recognize only fruits. It was difficult. Therefore, the present invention has the same hue component of fruits and leaves, such as cucumber,
Moreover, even if the fruit and the leaf overlap in the front and back, it is a subject to separate and identify each.

[0004]

In order to solve the above problems, the present invention has the following constitution. That is, a visual device such as a fruit harvesting robot according to the present invention is a camera that captures only an image of light of a specific hue component, a distance measuring element that measures a distance to an object, and a measurement result of the distance measuring element. It is characterized by comprising a CPU for analyzing an image obtained by the camera based on the above, and for distinguishing a foreign substance such as a fruit and a leaf overlapping with the fruit.

[0005]

First, the input signal of the camera is binarized to obtain an image in which a plurality of objects, for example, fruits and leaves overlap. Next, the distance to the object is measured by the distance measuring element, and the image is analyzed based on the measurement result. A plurality of objects are separated and identified by the distribution of distance values.

[0006]

[Embodiment] FIG. 3 shows a usage state of a fruit harvesting robot for cucumbers. This fruit harvesting robot 1 is provided with an electric traveling carriage 2 as a moving means, an inclined frame 4 having a guide rail 3 is installed on the traveling carriage, and a manipulator 6 for fruit harvesting and a visual device 7 are moved up and down. It is provided freely. The tilted frame 4 is pivotally supported by the traveling carriage 2 by a hinge 8 and supported on the back side by a support link 9. The lower end of the support link 9 is fixed to a proper position of the long hole 10, and the tilting frame 4 is changed by changing the fixing position.
The inclination angle of can be adjusted arbitrarily.

The manipulator 6 includes the guide rail 3
A base 12 that can be moved up and down along the base 12, a main body 13 that is rotatably provided on the base in a horizontal plane, an articulated arm 14 provided on the main body, and a tip end of the articulated arm. It is provided with the picking hand 15. The articulated arm 14 includes an upper arm part 17 on the main body side and a forearm part 18 on the fruit picking hand side of the elbow part 1.
It is connected via 9 and, like a human arm, the whole arm can be vertically rotated, bent and extended, and the fruit picking hand 15 can be vertically rotated. When the articulated arm 14 is bent, the elbow portion 19 is bent so as to come to the lower side. The fruit picking hand 15 is configured to grasp the fruit and cut the fruit handle.

As shown in the block diagram of FIG. 1, the visual device 7 includes an image sensor camera (CCD) 31 for picking up only a hue component of green and a distance measuring element (PSD) 32 for measuring a distance to an object. The visual part 7a and the CPU 33 that analyzes and processes input signals from these visual parts. The visual part 7a is provided at the upper end of the manipulator main body 13, and the CPU 33 is built in the main body 13.

The distance detecting element 32 has the structure shown in FIG. The light emitted from the light projector 40 is reflected by the reflecting mirror 41 to illuminate the object 42, and the reflected light is reflected by the reflecting mirror 4
The light is reflected by 3 and received by the light receiver 44. Reflector 41,
Since the distance 43 is constant, the distance to the object 42 is measured based on the principle of triangulation. Reflecting mirrors 41, 4
Horizontal scanning is performed by changing the angle of 3 to the stepping motors M ₁ and M _2, and vertical scanning is performed by rotating the entire case 46 by the stepping motor M ₃ .

Before starting work, the visual device 7 is adjusted by the following method (see FIG. 4). First, as shown in FIG. 5, a plate 48 having a prescribed pattern 47 is provided in front of the camera 31, and the camera 31 takes an image. Next, the distance measuring element 32 uses the distance values d (1 to 1) of the four corner points (Q ₁ , Q ₂ , Q ₃ , Q ₄ ) of the specified pattern 47.
4) is measured and the stepping motors M ₁ , M ₂ and M ₃ are driven so that they are equal. As a result, the image input range of the camera 31 and the measurement range of the distance measuring element 32 match. The scanning angle at this time is stored.

The processing for searching for fruits is performed in the order shown in the flowchart of FIG. In the illustrated example, the resolution of the input signal level is 64. The camera 31 adjusts the input signal level to be less than 64 in all pixels.
The aperture is adjusted and the threshold value is calculated from the average value of the input signal level.
Quantify. Then, noise in the image is removed. It is determined whether or not the number of "1" pixels after noise removal is equal to or larger than a predetermined value P. If the number is equal to or larger than the predetermined value P, the camera 31 catches something and proceeds to the next step. The visual part 7a is moved on the assumption that nothing is reflected on the camera 31.
When the number of "1" pixels is equal to or larger than the predetermined value P, the contour line of the binarized image is obtained, and it is further determined whether or not the object of the image is a fruit based on the ratio of vertical and horizontal pixels, and it is predicted that the object is a fruit. Scanning with the distance measuring element 32 only in case of If it is determined that it is not a fruit, the visual part 7a is moved. It should be noted that the immature fruit is removed as noise because the light reflection area is small. As described above, since the distance measuring element 32 scans only when there is a fruit in the image of the camera 31, the processing time for fruit recognition is shortened.

When the camera 31 catches the fruit, the distance measuring element 3 is input before the image signal is input to the CPU 33.
The central portion of the screen is horizontally scanned at 2 to find the shortest distance, and the visual portion 7a is moved back and forth so that the shortest distance is about 30 cm (see FIG. 7). Since the distance from the camera 31 to the object is always almost the same, it is possible to obtain information under the same condition where the influence of illumination is constant. In addition, because the size of the fruit can be roughly determined only by the number of fruit recognition pixels,
Processing time is shortened.

To determine the fruit size, the process shown in the flowchart of FIG. 8 is performed. First, the image of the image sensor camera 31 is binarized into a fruit portion and a background portion, and the center point C of the fruit portion is obtained (see FIG. 9). When the distance measuring element 32 horizontally and vertically scans the vicinity of the center point C and the distance value of the center point is dc, the distance value is dc.
The shape of the fruit is revealed by extracting the scanning points that are ± α. This allows only ripe fruits to be harvested.

It should be noted that the length and width of an image which is considered to be a fruit are calculated based on the pixel pitch, and this is compared with a comparison table prepared in advance to determine whether or not it is actually a fruit (see FIG. 10). ). For example, the standard relationship between length and width of a mature cucumber is shown in FIG. As described above, by confirming that the fruit is a fruit by utilizing the relationship between the length and the width (diameter), it is possible to prevent accidental damage to the stem at the time of harvesting.

When a plurality of objects are imaged in one screen, a histogram of distance value distribution of each image (see FIG. 13)
Are created, and leaves and fruits are identified by the frequency and area (see FIG. 12).

Further, as shown in FIG. 15, for example, leaves 50
If the fruit (cucumber) 51 overlaps the front and back,
Both are displayed simultaneously on one screen. When this happens,
Both are separated by performing the analysis shown in the flowchart of FIG. First, the image sensor camera 31 captures an object and binarizes it into a green hue component and other hue components. Since both the fruit and the leaf are green, the binarized image has a shape in which the both are combined as shown in FIG. Next, the contour line of this binarized image is obtained, and the distance detection element 32
Scan horizontally and vertically. Then, the difference between the distance values on the contour line is calculated, and as shown in FIG. 17, with reference to the threshold value L, a line segment A having a distance value smaller than the threshold value L and a distance value larger than the threshold value L. The contour line is separated into line segment B. The line segment A represents the outline of leaves, and the line segment B represents the outline of cucumbers.

In the above analysis procedure, instead of taking the difference between the distance values on the contour line, a histogram of the distance values within the contour line is created as shown in FIG. 19, and the contour line of the leaf 50 and the fruit 51 is created by the histogram. May be separated (see FIG. 18).

When the visual device 7 finds a harvestable fruit, the manipulator 6 performs a predetermined operation to harvest the fruit. When all the fruits within the visual field of the visual part 7a have been harvested, the direction of the visual part 7a is changed and the harvesting of the fruits is repeated.
When all the fruits within the entire working range of the manipulator 6 have been harvested, the traveling carriage 2 is moved to move to the next stock.

[0019]

As described above, the visual device of the fruit harvesting robot according to the present invention analyzes the image taken by the camera based on the measurement result of the distance measuring element to detect leaves and fruits such as cucumber. Even for crops with the same color,
The two can now be reliably separated and identified.

[Brief description of drawings]

FIG. 1 is a block of a visual device.

FIG. 2 is a diagram showing a configuration of a distance measuring element.

FIG. 3 is a diagram showing a usage state of a fruit harvesting robot.

FIG. 4 is a flow chart of first visual device control.

FIG. 5 is a diagram showing a concept of first visual device control.

FIG. 6 is a flowchart of second visual device control.

FIG. 7 is a flowchart of third visual device control.

FIG. 8 is a flow chart of fourth visual device control.

FIG. 9 is a diagram of an image for supplementing the explanation of the fourth visual device control.

FIG. 10 is a flowchart of fifth visual device control.

FIG. 11 shows the standard length-diameter relationship for mature cucumbers.

FIG. 12 is a flowchart of sixth visual device control.

FIG. 13 is a histogram of distance values to supplement the description of the sixth visual device control.

FIG. 14 is a flowchart of seventh visual device control.

FIG. 15 is a diagram showing a concept of seventh visual device control.

FIG. 16 is a diagram of an image for supplementing the explanation of the seventh visual device control.

FIG. 17 is a diagram showing a relationship between a contour line and a difference for supplementing the explanation of the seventh visual device control.

FIG. 18 is a flow chart of eighth visual device control.

FIG. 19 is a histogram of distance values to supplement the description of the eighth visual device control.

[Explanation of symbols]

 1 Fruit Harvesting Robot 6 Manipulator 7 Visual Device 15 Fruit Picking Hand 31 Image Sensor Camera 32 Distance Detection Element 33 CPU 50 Leaf 51 Fruit

Claims (1)

[Claims] 1. A camera for capturing only an image of light of a specific hue component, a distance measuring element for measuring a distance to an object, and an image by the camera is analyzed based on a measurement result of the distance measuring element. , A visual device such as a fruit harvesting robot having a CPU for identifying a fruit and a foreign substance such as a leaf overlapping with the fruit.