JP3336663B2 - Visual devices such as fruit harvesting robots - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual search device provided in a fruit and vegetable harvesting robot for searching for crops.

[0002]

2. Description of the Related Art Fruit and vegetable harvesting robots have been developed which identify mature fruits and automatically harvest while self-propelled under the vegetation where fruits have been cultivated.・ Two-panel image sensor camera (CCD camera) to distinguish fruits by using a point that has a specific tendency in the change in reflectance corresponding to the change in light wavelength depending on each part of the plant such as leaves and stems And calculates the luminance of each wavelength given to the pixel by the two different wavelengths, and calculates a pixel having a value of the processing value equal to or greater than a certain value and a pixel having a value equal to or less than a certain value.
Techniques have been developed for selecting a fruit using a digitized image and determining its shape.

[0003]

When the shape of the fruit is extracted in this way, the brightness of the specular reflection portion of the fruit at the two wavelengths is both large and shows the same brightness level.
Since a calculation value indicating the feature of the fruit cannot be obtained, it becomes a missing portion in the image, and the shape of the fruit cannot be accurately grasped.

[0004]

According to the present invention, there is provided a camera for photographing an object with a plurality of lights having different wavelengths. The brightness ratio of each wavelength is calculated using a predetermined arithmetic expression, and the shape is determined while selecting an object as an image obtained by binarizing the calculated value into a portion equal to or more than a certain value and a portion less than a certain value. In the visual device for detecting, the specular reflection part of the object which causes the loss in the image is divided into two parts: a part where the luminance of the image of the wavelength at which the specular reflection part is detected particularly strongly, and a part where the luminance is less than the constant. A visual device, such as a fruit and vegetable harvesting robot, characterized in that the visual device is supplemented with a converted image.

[0005]

[Operation and Effect of the Invention] The specular reflection part of the fruit, which has become a missing part in the calculation image, is used in an image of a wavelength which particularly strongly senses the specular reflection part of an image taken at a plurality of wavelengths.
If a binarized image of the wavelength is created by setting a threshold value at a high luminance level, this image will be an image showing only the specular reflection portion, and if this image is superimposed on the calculated image, a complete image of the fruit body will be obtained. Can be obtained, and the shape, dimensions, and the like can be accurately detected.

[0006]

1 to 4 show a cucumber harvesting robot according to an embodiment of the present invention.
Is provided with an electric traveling unit 2 as a moving means, and a manipulator 3 for harvesting fruits and vegetables, a visual device 5 for searching for a crop, and the like on the traveling unit. In a cultivation field using the fruit and vegetable harvesting robot 1, the cucumber tree 8 is supported on the support 7 provided diagonally and grown. The manipulator 3 includes an inclined frame 10 inclined so as to be substantially parallel to the inclination of the facing tree body 8, a base 12 mounted to be able to move up and down along a guide rail 11 of the inclined frame, and The body comprises a main body 13 rotatably provided in a horizontal plane, an articulated arm 14 provided on the main body, and a thinning hand unit 15 provided at the tip of the arm. The inclined frame 10 is pivotally supported by the traveling section 2 using a hinge 16, and the back side is supported by a support link 17. The lower end of the support link 17 is fixed to an arbitrary position of the long hole 18, and the angle of inclination of the inclined frame 10 can be arbitrarily adjusted by adjusting the fixing position. By operating each part of the manipulator 3, the hand 19 is configured to hold the fruit 19 as a target and cut the fruit handle. As shown in FIG. 2, the visual device 5 includes a multi-plate CCD camera 6, a distance sensor (PSD) 20 that scans the field of view of the camera horizontally and vertically to measure a distance to an object, and And a memory 4 for storing data.

A visual device 5 comprising a multi-plate CCD camera 6 (FIGS. 3 and 4) and a PSD 20 is installed above the main body 13 of the manipulator. The light condensed by the lens 22 passes through the prism 23, is refracted by 90 ° in the direction, is filtered by the filter 27, and forms an image on the light receiving element plate 28 with monochromatic light of a predetermined wavelength.

Filters 27 and 2 having different wavelength bands to be filtered
9 and the light receiving element plates 28 and 30 form a light receiving section 35, 36 on a plane X perpendicular to the optical axis Y of the lens 22.
The prism 23 is disposed at an equal distance from the optical axis Y on the
At 5, rotation is made about the optical axis Y, and light is sent to each of the light receiving portions 35 and 36 to form an image on the light receiving element plates 28 and 30.

The image connected to the light receiving portions 35 and 36 can be identified by the rotation position of the rotating prism 23 as to which image the light receiving portions 35 and 36 are. In the same manner as described above, the light receiving section 37 having the filter 33 and the light receiving element plate 34 may be arranged on the plane X. A fruit discriminating method of the visual device 5 of the fruit harvesting robot 1 having such a configuration will be described by taking discrimination of cucumber fruit as an example.

FIG. 5 shows the reflectance according to the wavelength of light that strikes the fruits 19, flowers, leaves and stems of the cucumber tree 8,
It shows that at the wavelengths of 550 nm (nanometers) and 850 nm, the reflectance tendencies of fruits and other parts are different. That is, the reflectance of the cucumber fruit 19 has a wavelength of 8
It is the highest at 50 nm compared to other parts, and has a wavelength of 55 nm.
It has the lowest tendency at 0 nm.

Now, using the two-chip CCD camera 6, the filter 37 of the light receiving unit 35 is made to pass only light near the wavelength of 850 nm, and the filter 29 of the light receiving unit 36 is set to the wavelength 550.
If only light in the vicinity of nm is allowed to pass, the light receiving elements are arranged in a grid pattern, and the intensity of light projected on each light receiving element is detected to detect the projected image. The cucumber fruit image projected on the light receiving element plate 28 has higher luminance than other parts such as stems and leaves, and the cucumber fruit image projected on the light receiving element plate 30 has lower luminance than other parts. .

Using this characteristic, the intensity N850 of monochromatic light having a wavelength of 850 nm given to the element on the light receiving element plate 28 at the same point of the image is given, and the light intensity given to the corresponding element on the light receiving element plate 30 is obtained. Assuming that N550 is N550, a calculation formula (example of calculation formula λ = N850 / N85) determined to promote the characteristics
0 + N550), the ratio λ of the light intensity at both wavelengths
Is calculated.

[0013] The light intensity ratio λ is calculated for all points on the captured screen, and the calculated value is binarized using a threshold value to form a calculated image of the fruit 19. Since the value of λ in the example formula is larger in the number of pixels to be performed than in other parts, a clearer image can be obtained because more points are taken in than in other parts such as stems and leaves. However, among the pixels constituting the image D1 of the cucumber fruit 19, the glossy portion D2 of the mirror surface has a higher reflectance for all wavelengths than the portion outside the mirror surface, and the values of N850 and N550 are almost equal, and Since the value becomes smaller in the example formula, it is deleted by the threshold value, and the image D2 of this portion is lost (FIG. 7-A).

Since the above-described specular reflection portion D2 shows a particularly high value among the N850 image data, the N850 image data is partitioned by a high threshold value, and if a binarized image is formed, the specular reflection portion D2 is formed. Image D3 can be obtained (FIG. 7-B). Now, on the above-mentioned computed image D2 (FIG. 7-A), an image D3 (FIG.
-B), the shape D of the fruit 19 can be accurately drawn without any missing parts (FIG. 7-C), and the visual device 5 of the robot can clearly grasp the shape and size of the fruit 19. it can. E, E1, E2, and E3 are images of leaves. In the flowchart of another embodiment shown in FIG. 8, in rendering the specular reflection portion, when binarizing N850, the number of recognition pixels, from the higher luminance level by 1 / n (n> 2) of the number of recognized pixels By selecting and binarizing this, the processing time is shortened without damaging the image of the specular reflection part. The CPU processing pays attention to the fact that the specular reflection part shows a higher reflectance than other parts. This is to reduce the load.

FIG. 9 shows another embodiment in which the wavelength is 850 nm.
Light emitting device 31 for irradiating light of wavelength 550 nm
A strobe light emitting device 32 for irradiating light of the same color and a two-chip CCD camera 6 are provided to obtain clear monochromatic light images irrespective of the amount of sunlight due to the weather and the brightness of day and night in synchronization with the irradiation timing of light of each wavelength. By doing so, it is not necessary to increase the level difference between the amount of light collected from the fruit 19 and the stem / leaf, etc., and to finely change the threshold value of the calculated value according to the lighting conditions.
The threshold value is set to an approximate value, and the arithmetic processing is simplified.

FIG. 10 shows the energy distribution of light emitted by the strobes 31 and 32 for each wavelength.

[Brief description of the drawings]

FIG. 1 is a front view of a fruit harvesting robot in a fruit growing field.

FIG. 2 is a block diagram of a fruit harvesting robot.

FIG. 3 is a plan sectional view of a multi-plate CCD camera.

FIG. 4 is a partial front sectional view thereof.

FIG. 5 is a graph showing the reflectance of each part of the cucumber tree with respect to the wavelength of light.

FIG. 6 is a flowchart of image processing.

FIG. 7 is a processed image.

FIG. 8 is a flowchart of another embodiment.

FIG. 9 is a perspective view of a camera provided with a light emitting device of another embodiment.

FIG. 10 is a wavelength distribution diagram of light emission energy of the light emitting device.

[Explanation of symbols]

 Reference Signs List 1 fruit harvesting robot 5 visual device 6 multi-plate CCD camera 21 CPU 27 filter 28 light receiving element plate 29 filter 30 light receiving element plate 35 light receiving unit 36 light receiving unit

Claims (1)

(57) [Claims] 1. A camera for photographing an object with a plurality of lights having different wavelengths, wherein a predetermined arithmetic expression is used for each pixel based on the luminance of a plurality of wavelengths given to each pixel by an image photographed by the camera. A visual device that calculates the ratio of the luminance of each wavelength and binarizes the calculated value into a portion equal to or greater than a certain value and a portion less than a certain value as an image while detecting the shape of an object while selecting the object. The specular reflection portion of the object causing the missing portion is supplemented by a binarized image of a portion where the luminance of an image having a wavelength at which the specular reflection portion is detected particularly strongly is a portion where the luminance is equal to or higher than a certain value and a portion where the luminance is lower than a predetermined value. A visual device such as a fruit vegetable harvesting robot.