JPH05174130A - Visual device for fruit havest robot - Google Patents

Abstract

PURPOSE:To improve the detection accurately of fruit and leaves by controlling the light quantity of a lighting device corresponding to the distance between a camera and an object as well as lightness. CONSTITUTION:A camera 20 photographing an object, a distance sensor 21 measuring the distance to the object, a lightness measurement means 26 measuring the lightness around the object, a lighting device 22 emitting light to the object, and a controller 23 controlling the light amount of the lighting device 22 corresponding to the distance between the camera 20 to the object which is measured by the distance sensor 21 and the lightness which is measured by the lightness measurement means 26, are provided. The distance between the camera 20 and the object is measured by the distance sensor 21, and the light quantity of the lighting device 22 is increased when the distance is too far or the light amount of the lighting device 22 is reduced when the distance is too close. The lightness around the object is measured by the lightness measurement device 26, and the light quantity of the lighting device 22 is increased when the lightness is too low or the light amount of the device 22 is reduced when the lightness is too high. Thus, the entire light quantity in a camera image can be always properly adjusted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual device for searching a harvested product provided in a fruit harvesting robot.

[0002]

[Prior Art] While self-propelled along the planting line of cultivated crops,
A fruit harvesting robot has been developed that automatically harvests mature fruits. A visual device for searching a harvested product provided in a fruit harvesting robot of this type binarizes an image captured by an image sensor camera into a portion having a brightness equal to or higher than a certain value and a portion having a brightness less than a certain value, and outputs a fruit or the like from the image. It is designed to extract the outline of.

[0003]

The above-mentioned conventional visual device has a problem that the amount of light is insufficient when it is cloudy or rainy, which may make it impossible to detect fruits or leaves.

[0004]

In order to solve the above problems, the present invention has the following constitution. That is, the visual device of the fruit harvesting robot according to the present invention, a camera for imaging the object, a distance sensor for measuring the distance to the object, a brightness measuring means for measuring the brightness of the surroundings of the object,
A lighting device that illuminates an object, and a control device that controls the light amount of the lighting device according to the distance from the camera measured by the distance sensor to the object and the brightness measured by the brightness measuring device. It is characterized by having.

[0005]

The distance from the camera to the object is measured by the distance sensor, and the light quantity of the illuminator is increased when the distance is far, and the light quantity of the illuminator is decreased when the distance is close. Further, the brightness around the object is measured by the brightness measuring means, and the light quantity of the illuminator is increased when it is dark, and the light quantity of the illuminator is decreased when it is bright. In this way, by controlling the light quantity of the illuminator according to the distance from the camera to the object measured by the distance sensor and the brightness measured by the brightness measuring means, the total light quantity in the camera image is always properly adjusted. It is possible to improve the detection accuracy of fruits and leaves.

[0006]

[Embodiment] FIG. 2 shows a usage state of a fruit harvesting robot for cucumbers. The fruit harvesting robot 1 includes an electric traveling unit 2 as a moving unit, and a manipulator 3 for fruit harvesting, a visual device 5 for searching for a harvested product, and the like are installed on the traveling unit. In a cultivating field where the fruit harvesting robot 1 is used, a cucumber tree 8 is supported by a support 7 provided obliquely.

The manipulator 3 includes an inclined frame 10 which is inclined so as to be substantially parallel to the inclination of the tree 8 which faces it, a base 12 which is vertically movable along a guide rail 11 of the inclined frame, and the base. A main body 13 rotatably provided on a table in a horizontal plane, an articulated arm 14 provided on the main body,
It comprises a fruit picking hand portion 15 provided at the tip of the articulated arm. The tilted frame 10 is pivotally supported on the traveling unit 2 by a hinge 16, and the back side thereof is supported by a support link 17. The lower end of the support link 17 is fixed to a proper position of the long hole 18, and the tilt frame 1 is changed by changing the fixing position.
The tilt angle of 0 can be adjusted arbitrarily. By appropriately operating each part of the manipulator 3, the hand part 15 is brought close to the target fruit. Fruit picking hand part 15
Is configured to grasp a fruit and cut its peduncle.

As shown in the block diagram of FIG. 1, the visual device 5 includes an image sensor camera 20 and a distance sensor (PSD) 21 that horizontally and vertically scans the field of view of the camera to measure the distance to an object. And a strobe 22 as an illuminating device for irradiating the object with light, the CPU 23 processes the input signals of the camera 20 and the PSD 21, and determines the position of the harvestable fruit and the appropriate amount of light of the strobe 22. Then, based on the determination result, an output signal is output to the manipulator 3 and the strobe 22. The data in the CPU 23 is stored in the memory 24.

The control for detecting the fruit position is performed in the order shown in the flowchart of FIG. First, the image of the camera 20 is input, and the range of the screen is horizontally and vertically scanned by the PSD 21 to measure the distance to the object. Next, the camera image is binarized based on the brightness, and the contour of the object is extracted. For example, if the camera image is as shown in FIG.
If the luminance distribution of the pixels on the line is as shown in FIG. 5,
Pixels whose brightness is equal to or higher than the binarization level (hatched portion in FIG. 5) are processed as "1", and pixels whose brightness is equal to or lower than the binarization level are processed as "0" to obtain the processing screen shown in FIG. The ratio of the vertical length and the horizontal width of the object on this processing screen is obtained, and if the ratio is within the specified range, the object is determined to be cucumber. Then, the coordinates of the center point of the cucumber are obtained, and the manipulator 3 is moved so that the hand unit 15 moves toward the center point.
To drive.

The illumination intensity of the strobe 22 is determined by the control shown in the flowchart of FIG. First, the image of the camera 20 is input with the illumination off. Then, PSD2
1 scans the object in the camera field of view horizontally and vertically,
Calculate the average distance to the object. Next, a brightness histogram is created based on the brightness of each pixel of the camera image, and the center of gravity of the histogram is obtained. For example, as shown in FIG. 8, the barycentric brightness value b _{1 in} fine weather (solid line) is large,
The center-of-gravity luminance value b _{2 under} cloudy weather (broken line) is small. The illumination intensity of the illuminator 22 is determined based on the distance value and the brightness value thus obtained. Generally, the illumination intensity is increased as the distance to the object is increased, and the illumination intensity is increased as the surrounding area of the object is dark. Table 1 is an example of illumination intensity adjustment. In this example, distance and brightness are divided into three stages of large, medium, and small, respectively, and illumination intensity is divided into three stages of strong, medium, and weak.

[0011]

[Table 1]

When the average luminance of the image input by the camera is remarkably reduced, it is considered that an abnormality has occurred in the visual device. Therefore, in order to detect the abnormality, the control shown in the flowchart of FIG. 9 is performed. The center-of-gravity brightness value used in the control may be used instead of the average brightness. The specified value is determined empirically. If it is determined that an abnormality has occurred in the visual device, work is stopped or an operator is notified by an alarm device.

Next, different control will be described.

When the natural light level is too low at the time of camera input, the resolution of the camera becomes lower than the limit, and the object cannot be detected. The difference becomes small and the object cannot be detected accurately. When the strobe light 22 is used to illuminate the target object, the brightness of the nearby fruits will increase,
The difference between the brightness level of the object and the brightness level of the background portion is increased. As it is, the whole brightness is too high and becomes oversaturated. Therefore, by comparing the data when the light is projected and the data when the light is not projected, only the data of the object is emphasized and the object can be accurately detected. .. FIG. 10 is a flowchart of the control.

Instead of measuring the brightness around the object from the brightness of the camera image, the brightness in front of the camera 20 may be measured by a separately provided optical sensor 26. In that case, the illumination intensity of the strobe 22 is adjusted by the control shown in the flowchart of FIG.

[0016]

As described above, the visual device for the fruit harvesting robot according to the present invention illuminates according to the distance from the camera to the object measured by the distance sensor and the brightness measured by the brightness measuring means. By controlling the amount of light of the ingredients, the total amount of light in the camera image can always be made appropriate, and the detection accuracy of fruits and leaves has improved.

[Brief description of drawings]

FIG. 1 is a block of a visual device.

FIG. 2 is a diagram showing a usage state of a fruit harvesting robot.

FIG. 3 is a flowchart of first visual device control.

FIG. 4 is a diagram of a camera input image.

FIG. 5 is a diagram showing a luminance distribution.

FIG. 6 is a diagram of a processed image.

FIG. 7 is a flow chart of second visual device control.

FIG. 8 is a diagram of a luminance histogram.

FIG. 9 is a diagram of an image for supplementing the explanation of the third visual device control.

FIG. 10 is a diagram of an image for supplementing the explanation of the fourth visual device control.

FIG. 11 is a diagram of an image for supplementing the explanation of the fifth visual device control.

[Explanation of symbols]

 1 Fruit Harvesting Robot 3 Manipulator 5 Visual Device 15 Hand for Fruit Picking 20 Camera 21 Distance Sensor 22 Strobe (Lighting Equipment) 23 CPU (Control Device)

Claims (1)

[Claims] 1. A camera for capturing an image of an object, a distance sensor for measuring a distance to the object, a brightness measuring unit for measuring brightness around the object, and an illuminator for irradiating the object with light. A fruit that is provided with a control device that controls the light amount of the lighting device according to the distance measured from the camera to the object by the distance sensor and the brightness measured by the brightness measuring device. Visual device for harvesting robot.