JP2784585B2 - Fruit and vegetable position recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to an apparatus for recognizing fruit and vegetable positions when harvesting fruit and vegetables such as fruits and vegetables using a manipulator such as a robot.

[Conventional technology]

As a device for recognizing a fruit position in a fruit harvesting robot that harvests fruits such as apples, those using the following two methods are conventionally known. That is, two cameras or camera 1
A method for recognizing fruit positions by stereo ranging from two images obtained by moving the platform by a fixed distance, and a method for recognizing fruit positions by obtaining a plurality of images by visual feedback from a camera provided at the tip of a robot hand It is.

[Problems to be solved by the invention]

Generally, image processing requires an operation for each pixel,
The amount of calculation becomes enormous, and when a plurality of images are image-processed to find corresponding points included in each image and the position of the target fruit is recognized as in the above two conventional techniques, the amount of calculation becomes further enormous. A long processing time is required, and an expensive computing device having a high-speed computing capability is required to reduce the processing time, and it has been difficult to put it to practical use.

The present invention has been made in view of such circumstances,
A light spot is applied to the center of gravity of the target fruit, and the center of gravity is imaged at a predetermined position in the field of view of the color camera, and triangulation is performed based on the optical axis of the spot light at that time and the optical axis of the incident light on the imaging means. By recognizing the position of the fruit by the method,
It is an object of the present invention to provide a fruit and vegetable position recognizing device that can reduce the amount of image processing, reduce image processing time, and reduce the time required for recognizing a fruit position.

[Means for solving the problem]

A fruit and vegetable position recognition device according to the present invention is a fruit and vegetable position recognition device that images a fruit to be harvested by a movable imaging unit, processes the obtained image, and recognizes the position thereof. Light emitting means for irradiating light, moving means for moving the light emitting means, means for calculating the position of the center of gravity of the imaged fruit or vegetable, and the moving means for irradiating the spot light to the calculated center of gravity And means for determining whether the spot light matches the center of gravity of the fruit and vegetable, and when the means determines that the spot light and the center of gravity of the fruit and vegetable match, Means for controlling the position of the imaging means so as to image the spot light at a predetermined position within the imaging field of view of the imaging means; and the direction of the optical axis of the light incident on the imaging means and the spot light from the light projecting means. Based on the direction of the optical axis Characterized in that it comprises a means for calculating the position of the serial fruit vegetables thereof.

[Action]

In the present invention, when the fruit or vegetable is detected by the imaging means, the position of the center of gravity is calculated, the moving means is controlled so that the light spot is irradiated on the center of gravity, and the light spot position is set to a predetermined position in the visual field of the imaging means. The imaging means is positioned in order to recognize the position of the fruit or vegetable based on the direction of the optical axis of the imaging means at that time and the direction of the optical axis of the spot light from the light projecting means.

〔Example〕

Hereinafter, the present invention will be described with reference to the drawings showing one embodiment. FIG. 1 is a perspective view showing an appearance of a fruit harvesting robot equipped with a fruit and vegetable position recognition device according to the present invention.

In the figure, reference numeral 1 denotes a vehicle body on which a harvesting manipulator 6 is mounted, and the vehicle body 1 is automatically steered by two driving wheels 2 and one steering wheel 3. A steering control unit 4 and a harvest control unit 5 are mounted on the rear part of the vehicle body 1.

A manipulator 6 is mounted on the upper center of the vehicle body 1. The manipulator 6 is a five-axis articulated type, and a hand 7 for catching the fruit 10 is attached to the tip of the arm. The hand 7 has a harvesting unit 8 that can be opened and closed by a hand actuator 17 composed of a motor.
Is controlled to be a predetermined value.

Further, a spot light irradiator 9 for irradiating a spot light is attached to the hand 7. These manipulators 6
The position and direction of the hand 7 attached to the robot 7 are detected by the hand posture detector 12 based on the detection values from the respective rotary encoders attached to the robot actuator 16 composed of five motors for driving the respective axes of the manipulator 6. ing. That is, the position of the hand 7 is obtained from the detection values of the rotary encoders of the turning part, the shoulder part, and the elbow part of the manipulator 6, and the turning angle and the tilt angle of the hand 7 are obtained from the rotary encoders of the two motors of the wrist part. Thus, the direction of the hand 7 is obtained.

A color video camera serving as an image pickup means is provided at a front portion of the vehicle body 1.
11 is mounted so that it can be swiveled and tilted by a camera actuator 13 composed of a motor with a rotary encoder of two bogies. The turning angle and the tilt angle are detected by a rotary encoder, and a camera direction detection unit 14 is Detects the direction of the camera.

FIG. 2 is a block diagram showing a schematic configuration of the harvest control unit. The harvest control unit 5 uses a 16-bit CPU, and its input terminals include an NTSC video signal from a color video camera and a pressure sensor 15. Pressure signal, camera direction detector 14
The direction of the color video camera 11 from the camera and the position and the direction of the hand 7 from the hand posture detecting unit 12 are input, and from the output terminals thereof, the respective robot actuators 16, the hand actuators 17 and the camera actuators 13 are separately driven. An output signal is output.

Next, the operation of the apparatus of the present invention configured as described above will be described. FIG. 3 is a principle diagram showing a procedure for recognizing a fruit position according to the present invention, and FIGS. 4 (a) and 4 (b) are flowcharts thereof.

In the apparatus of the present invention, first, as shown in FIG. 3, a fruit 10 to be harvested is detected by binarizing a captured image (FIG. 3 (a)), the center of gravity is calculated, and a spotlight is calculated there. Is controlled (FIG. 3 (b)), and then the camera actuator 13 is controlled so that the position of the spot light is at the center of the field of view (FIG. 3 (c)). The direction of the color video camera 11 and the posture of the hand 7 at that time are detected, and the distance between the manipulator 6 and the fruit 10 is obtained by a known method such as triangulation. Thereby, the movement amount and the direction of the manipulator 6 for harvesting the fruit 10 are calculated, and the fruit can be harvested.

 Next, the operation will be described in detail with reference to a flowchart.

First, imaging is started by the color video camera 11 (step 1), the image is input into the harvest control unit 5, and a pixel in which the red R is equal to or more than the predetermined threshold value R _S is extracted and binary. (Step 2). Next, the number of binarized pixels is counted, and it is determined whether or not the number is equal to or more than a predetermined value P _S (step 3). If the number is less than P _S , the camera actuator 13 is turned and tilted by a predetermined amount. Return to P When _S is greater than the determined area by the number of pixels (step 5), and calculates the barycentric position based thereon (Step 6). Next, the spot light irradiator 9 is turned on to irradiate a spot light (step 7). Then, an image is picked up by the color video camera 11 (step 8), and the image is binarized by a predetermined second threshold to extract a spotlight (step 9). As a result of the binarization, it is determined whether or not the spot light is within the imaging visual field (step 1).
0) If the spot light is not in the visual field, the manipulator 6 is driven by a predetermined amount (step 11), and the process returns to step 8.
When the spot light is within the field of view, the shift amount between the center of the spot light and the position of the center of gravity is calculated (step 12). Here, the shift amount includes the distance and the direction. When the shift amount is calculated, the manipulator 6 is driven according to the calculated amount (step 13), and the image is taken and binarized (steps 14, 1).
5) Match the spot light to the position of the center of gravity. Step 16
Then, it is determined whether or not the center of the spot light coincides with the position of the center of gravity. If not, the process is repeated from step 12 again. If they match, the amount of deviation between the position of the center of gravity and the center of the field of view is calculated (step 17), and the camera actuator 13 is driven based on the calculated amount (step 18). judge. If they do not match, the process repeats from step 17 again. If they match, the spotlight irradiating unit 9 is turned off immediately and the hand posture detecting unit 12
Calculates the position of the hand 7 from the detected values of the three rotary encoders of the turning part, the shoulder and the elbow of the robot actuator 16 of the manipulator 6, and calculates the direction of the hand 7 from the detected values of the two rotary encoders of the wrist. At the same time, the direction of the color video camera 11 is calculated from the detection values of the two rotary encoders attached to the camera actuator 13 (Step 20).

At this time, the center of the optical axis of the color video camera 11 (=
Since both the spot light from the center of the field of view and the spot light from the spot light irradiation unit 9 are located at the center of gravity of the fruit 10, the optical axis and the optical axis of the spot light are on the same plane, and a triangle formed by the two optical axes Is required uniquely. Then, the position of the fruit 10 is calculated using a known triangulation method.
The distance and direction between the hand and the hand 7 can be obtained (step 21).

By using these data, the manipulator 6 is driven to bring the hand 7 close to the fruit 10 to be harvested, and the hand actuator 17 of the harvesting unit 8 is opened and closed to harvest the fruit 10.

In this embodiment, an articulated robot is used as the manipulator.However, the present invention is not limited to this, and the manipulator may use any type of robot as long as it can automatically capture fruit and vegetables. Good.

Further, according to the configuration of this embodiment, not only the method of the present invention but also the conventional method of visual feedback and the method of stereo ranging can be used, and the position recognition method can be properly used depending on the target crop and the environment. High versatility.

〔The invention's effect〕

As described above, the position of the center of gravity is obtained by processing the image obtained by imaging the fruit and vegetable, the spot light is superimposed on the position of the center of gravity, and further, this is captured at a predetermined position in the field of view, and imaging is performed easily. The triangulation method enables easy and accurate recognition of fruit and vegetable positions, reduces the amount of calculation processing for detecting corresponding points, speeds up position recognition, and performs image processing with an inexpensive calculation device. It has excellent effects such as being able to.

[Brief description of the drawings]

FIG. 1 is a perspective view showing the appearance of a fruit harvesting robot equipped with a fruit and vegetable position recognition device according to the present invention, FIG. 2 is a block diagram showing the configuration of a harvest control unit, and FIG. 4 (a) and 4 (b) are flowcharts showing the principle. DESCRIPTION OF SYMBOLS 1 ... Body, 5 ... Harvest control part, 6 ... Manipulator, 7 ... Hand, 8 ... Harvest part, 9 ... Spot light irradiation part, 10 ... Fruit, 11 ... Color video camera, 12 ... …
Hand posture detector, 13 …… Camera actuator, 14…
… Camera direction detector, 16 …… Robot actuator

Claims (1)

(57) [Claims] 1. A fruit and vegetable position recognizing device which images a fruit to be harvested by a movable imaging means, processes the obtained image and recognizes the position thereof, and irradiates the fruit with spot light. Means, moving means for moving the light projecting means, means for calculating the position of the center of gravity of the imaged fruit and vegetables, means for controlling the moving means to irradiate the spot light to the calculated center of gravity, Means for judging whether or not the spot light matches the center of gravity of the fruit or vegetable; and if the means determines that the spot light and the center of gravity of the fruit or vegetable match, the image pickup means Means for controlling the position of the imaging means so as to capture an image at a predetermined position within the imaging field of view, based on the direction of the optical axis of light incident on the imaging means and the direction of the optical axis of spot light from the light projecting means Calculate the position of the fruits and vegetables Means for recognizing fruit and vegetable positions.