JP7174217B2 - Vegetable harvesting equipment - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fruit and vegetable harvesting apparatus, and more particularly, to a fruit and vegetable harvesting apparatus provided with harvesting means for gripping and harvesting stalks of fruits and vegetables.

2. Description of the Related Art Conventionally, in a fruit vegetable harvesting apparatus for harvesting vegetables such as strawberries and tomatoes whose edible parts (flower stalks in the case of strawberries) are easily damaged, the fruit stalks of the fruit vegetables are grasped at the time of harvesting, so that the fruit vegetables can be directly harvested. It is advisable not to touch it.
As such a fruit and vegetable harvesting apparatus, a harvesting means for gripping and harvesting the fruit stalk of the fruit and vegetables, a moving means for moving the harvesting means, a photographing means for photographing the fruits and vegetables, and based on the image photographed by the photographing means. and a control means for moving the collecting means by the moving means (Patent Document 1).
In the fruit and vegetable harvesting apparatus according to Patent Document 1, when the control means recognizes the fruit and vegetable from the image taken by the photographing means, the periphery of the fruit and vegetable is photographed again by the separately provided fruit pattern recognition camera, and the image is image-processed. By doing so, the fruit pattern of the fruit vegetable was recognized.
Further, in Patent Document 1, a direction is set for the collecting means to approach the fruit vegetables to be harvested so that the collecting means does not come into contact with the fruit vegetables not to be harvested, and the collecting means is moved in the set direction to is grasped.

Japanese Patent No. 5818240

However, when image recognition is performed using a fruit pattern recognition camera as in Patent Document 1, there is a problem that the fruit pattern cannot be accurately recognized by image processing if there is variation in the color of the fruit pattern.
In addition, in Patent Document 1, the collecting means is approached to the peduncle from a preset direction, but at this time, the position of the peduncle is not accurately recognized, and the peduncle is grasped at a rough position. was
In view of such problems, the present invention provides a fruit and vegetable harvesting apparatus capable of accurately recognizing fruit stalks and accurately recognizing the position of fruit stalks.

That is, in the fruit and vegetable harvesting apparatus according to the invention of claim 1, a harvesting means for gripping and harvesting the peduncle of the fruit and vegetables, a moving means for moving the harvesting means, a photographing means for photographing the fruits and vegetables, and the photographing means. A fruit and vegetable harvesting apparatus comprising control means for moving the picking means by the moving means based on a photographed image, and causing the picking means to grasp the fruit stalk of the fruit and vegetables,
An irradiating means for irradiating a line of light formed in a lateral direction with respect to the fruit stalk is provided above the photographing means, and the optical axis of the irradiating means intersects the photographing direction of the photographing means from obliquely above . set up as
The control means is characterized in that, from the image photographed by the photographing means, an irradiated portion of the fruit peduncle irradiated with the line light is detected, and the irradiated portion is recognized as the peduncle of the fruit vegetable.
Further, the fruit and vegetable harvesting apparatus according to the invention of claim 2 comprises a harvesting means for gripping and harvesting the peduncle of the fruit and vegetables, a moving means for moving the harvesting means, a photographing means for photographing the fruits and vegetables, and the photographing means. and a control means for moving the picking means by the moving means based on the image taken by the picking means so that the fruit stalk of the fruit vegetable is grasped by the picking means,
An irradiating means for irradiating a line of light formed in a lateral direction with respect to the fruit stalk is provided above the photographing means, and the optical axis of the irradiating means intersects the photographing direction of the photographing means from obliquely above . set up as
The control means, in the image photographed by the photographing means, detects an irradiated portion where the line light is irradiated to the fruit peduncle, recognizes the distance between the irradiated portion and a reference position set in the image, It is characterized by recognizing the positional relationship between the photographing means and the peduncle based on the distance.

According to the invention of claim 1, since the line light emitted by the lighting means is reflected on the fruit peduncle and photographed by the photographing means, it is possible to recognize this regardless of the color of the fruit peduncle. Become.
Further, by providing the optical axis of the lighting means so as to intersect the photographing direction of the photographing means, the line light passing behind the fruit stalk without being irradiated to the fruit stalk is prevented from being photographed by the photographing means. be able to.
According to the invention of claim 2, by providing the optical axis of the irradiating means so as to intersect the photographing direction of the photographing means, in the image photographed by the photographing means, the irradiated portion irradiated to the fruit peduncle is set in the image. By recognizing the distance from the reference position, it is possible to calculate the positional relationship between the photographing means and the fruit peduncle using trigonometry or the like, so that the harvesting means can be used to accurately harvest fruits and vegetables. .
Furthermore, according to the invention of claim 3, since the collecting means provided on the base member, the photographing means, and the irradiation means can be moved integrally, the photographing means and the irradiation means can be moved each time the fruit vegetable is recognized. This eliminates the need to re-recognize the positional relationship between , and the positional relationship between the photographing means and the sampling means, thereby enabling rapid processing.

Front view of harvesting equipment Side view of harvesting equipment A diagram showing an image photographed by a photographing means A diagram showing the positional relationship between the photographing means, the lighting means, and the strawberries.

1 and 2 show a strawberry harvesting device 2 for harvesting strawberries 1 as a fruit vegetable. A robot 5 is provided on the carriage 4 and harvests the strawberries 1 , and these are controlled by a control means 6 .
As shown in FIG. 2, the strawberry 1 is composed of an edible stem part 1a and a so-called calyx part 1b located above the stem part 1a. is formed.
When the strawberry 1 is harvested, the strawberry 1 is damaged if the flower stalk 1a is touched. Thus, strawberry 1 is harvested.
The cultivation bed 3 is adapted to cultivate strawberries 1 by hydroponics, for example, and is elongated in the horizontal direction of FIG. As shown in 2, the strawberry 1 hangs down on the side of the cultivation bed 3. As shown in FIG.
In addition to the strawberry 1, the fruit vegetables include fruits such as grapes, peaches, pears, and apples, and vegetables such as tomatoes, eggplants, cucumbers, and bitter melons. It is possible to harvest these fruit vegetables without damaging them.

The carriage 4 is reciprocated on rails 4a provided along the cultivation bed 3 by the driving force of a driving means (not shown). A tray 4b is provided for containing the strawberries 1 that have been placed.
The robot 5 has an arm portion 5a as a moving means fixed to a carriage 4 and a base member 5b provided at the tip of the arm portion 5a. A picking means 8 for picking strawberries 1, a photographing means 9 for photographing the strawberries 1, and an illumination means 10 for irradiating the stems 7 of the strawberries 1 with light are provided.
A conventionally known one can be used as the arm portion 5a. Although detailed description is omitted, by moving the base member 5b, the collecting means 8, the photographing means 9, and the lighting means 10 can be integrated. It is possible to move to

The collecting means 8 can use the structure described in Patent Document 1, and has a structure in which a pair of grippers 11 are opened and closed by driving means (not shown).
At the opposing portions of the gripper 11, a gripping portion 11a made of an elastic material for gripping the peduncle 7 from both sides, and a gripping portion 11a provided above the peduncle 11a to grip the peduncle 7 when the gripper 11 is closed. A cutting portion 11b for cutting is provided.
With such a configuration, when the picking means 8 is moved by the arm portion 5a and the peduncle 7 is positioned between the grippers 11, the grippers 11 are closed in that state. Then, the peduncle 7 is first gripped by the gripping portion 11a, and then the peduncle 7 is cut by the cutting portion 11b.
When the strawberry 1 is picked in this way, the picking means 8 is moved to the tray 4b provided on the carriage 4 by the arm 5a while gripping the fruit stalk 7 of the strawberry 1, and the strawberry 1 is picked up by the tray 4b. It is designed to be stored in an aligned state inside.

The photographing means 9 is fixed to the base member 5b of the robot 5, is positioned below the sampling means 8, and is provided so that its photographing direction is horizontal.
A CCD camera for photographing a color image may be used as the photographing means 9. Although one CCD camera may be used, a stereo camera comprising two CCD cameras may be used.

The illumination means 10 is fixed to the base member 5b of the robot 5, is positioned above the collection means 8, and is composed of a laser irradiation means for irradiating a line laser L formed in the lateral direction.
The length of the line laser L irradiated by the illumination means 10 is set to a length that crosses the photographing range photographed by the photographing means 9 from side to side, and the optical axis of the illumination means 10 is directed obliquely downward. is set towards.
In addition, the positional relationship between the lighting means 10 and the photographing means 9 and the direction of the optical axis of the line laser L emitted by the lighting means 10 are set so that the line laser L emitted by the lighting means 10 irradiates the fruit stem 7 of the strawberry 1. It is
Specifically, the optical axis of the line laser L irradiated by the irradiating means 10 is set to intersect the photographing direction of the photographing means 9 set horizontally, that is, the direction of the optical axis of the CCD camera from obliquely above. It is

The control means 6 recognizes the strawberry 1 to be harvested from the image photographed by the photographing means 9, recognizes the fruit stem 7 of the strawberry 1 to be harvested, and controls the collecting means 8 based on the recognition result. It is moved and the strawberry 1 is harvested.
Specifically, the control means 6 causes the strawberry 1 to be harvested based on the following steps.
First, the control means 6 performs a selection step of photographing a plurality of strawberries 1 by the photographing means 9 and selecting strawberries 1 to be harvested from among them.
The control means 6 moves the photographing means 9 to a position slightly separated from the side of the cultivation bed 3, and when the photographing means 9 photographs in this state, a plurality of strawberries 1 are reflected in the photographed image. It will happen.
The control means 6 recognizes the strawberries 1 to be harvested from among the plurality of strawberries 1 based on the size and color of the flowering part 1a.

Next, in order to harvest the recognized strawberry 1 to be harvested, the control means 6 performs an approach direction setting step of determining from which direction the collecting means 8 should approach the strawberry 1 .
In other words, when the strawberries 1 not to be harvested are located near the strawberries 1 to be harvested, the collecting means 8 may come into contact with the strawberries 1 not to be harvested depending on the direction in which the collecting means 8 is approached. This is because there is a risk of it being lost.
As for the selection step and the approach direction setting step, the processing described in Patent Document 1 can be used, and further description will be omitted.

Subsequently, the control means 6 performs a fruit pattern recognition step of recognizing the fruit pattern 7 of the strawberry 1 to be harvested from the image photographed by the photographing means 9 .
The control means 6 controls the arm portion 5a of the robot 5 to cause the collecting means 8 to approach the strawberry 1 to be harvested from the approach direction set in the approach direction setting step.
Subsequently, the photographing means 9 photographs the strawberry 1 at a position close to the strawberry 1 to be harvested, and the control means 6 recognizes the shape of the strawberry 1 from the photographed image, and the position of the center of gravity of the strawberry 1. to recognize
The control means 6 recognizes a predetermined position in the image photographed by the photographing means 9 as a reference position A, and moves the arm portion 5a so that the position of the center of gravity of the strawberry 1 coincides with the reference position A. The photographing means 9 is moved under control.
It should be noted that it is not always necessary to move the photographing means 9 in order to match the reference position A with the position of the center of gravity of the strawberry 1 . A reference position A may be set as the position.

When the position of the photographing means 9 is determined in this way, the control means 6 irradiates the line laser L from the illumination means 10, and the fruit stem 7 of the strawberry 1 is irradiated with the line laser L.
FIG. 3 shows an example of an image photographed by the photographing means 9, in which only the irradiated portion S of the line laser L irradiated to the fruit stalk 7 is photographed.
As described above, since the optical axis of the illumination means 10 intersects the photographing direction of the photographing means 9, the portion of the line laser L other than the portion irradiated to the fruit peduncle 7 is behind the peduncle 7. , and is not photographed by the photographing means 9.
Then, the control means 6 detects the irradiated portion S by the line laser L from the image photographed by the photographing means 9, and recognizes that the irradiated portion S is the fruit stalk 7 of the strawberry 1. It's becoming
Here, the control means 6 detects the irradiated portion S, which is the reflected light of the line laser L irradiated to the fruit peduncle 7, from the photographed image, and thus detects this regardless of the color of the peduncle 7. Therefore, it is possible to detect the fruit stalk 7 more reliably.

Here, when other strawberries 1 and leaves are located behind the strawberry 1 to be harvested, these are also irradiated with the line laser L, and the reflected light is also photographed by the photographing means 9.
Even in that case, since the peduncle 7 is thin, the irradiated portion S is photographed as a very small area. It can be recognized as the peduncle 7.
In addition, when the illuminating means 10 irradiates the line laser L obliquely downward from above the photographing means 9, and the fruit stalk 7 of the strawberry 1 to be harvested is closer to the photographing means 9 than the other strawberries and leaves. , the irradiated portion S irradiated to the fruit stalk 7 is positioned above the reflected light irradiated to the other portions.
Therefore, the control means 6 can recognize the reflected light detected in the uppermost portion of the image photographed by the photographing means 9 as the irradiated portion S indicating the fruit stalk 7 .
In the above-mentioned fruit stem recognition step, even if the light is not irradiated in a narrow range like the line laser L, by inclining the photographing direction and the optical axis, the light not irradiated to the fruit stem 7 is Since it does not pass backward and is photographed, the irradiated portion S can be recognized from the photographed image, and the fruit stalk 7 can be easily recognized.

After the fruit pattern recognition step is finished in this way, the control means 6 performs a picking position recognition step of recognizing the position of the fruit pattern 7 of the strawberry 1 to be harvested from the image photographed by the photographing means 9 .
4A and 4B are diagrams for explaining the positional relationship of the strawberry 1 with respect to the photographing means 9 and the lighting means 10. Of the images photographed by the photographing means 9 shown in FIG. 3, FIG. 3(b) shows an image of strawberry 1(a) approaching means 9, and an image of strawberry 1(b) separated from photographing means 9, respectively.
As can be understood from FIG. 4, when the strawberry 1(a) is approaching the photographing means 9, the line laser L is irradiated at a position spaced apart from the strawberry 1. ), the distance between the irradiated portion S indicating the fruit stalk 7 and the reference position A (gravity center position of the strawberry 1) set in the image increases.
On the other hand, when the strawberry 1(b) is separated from the photographing means 9, the line laser L irradiates a position close to the strawberry 1 from above. The distance between the irradiated portion S indicating the handle 7 and the reference position A becomes smaller.

Therefore, the control means 6 recognizes the distance between the irradiated portion S indicating the peduncle 7 and the reference position A set in the image, and determines the positional relationship between the photographing means 9 and the peduncle 7 based on the distance. recognize.
Specifically, since the positional relationship between the photographing means 9 and the lighting means 10 and the angle between the photographing direction of the photographing means 9 and the optical axis of the lighting means 10 are constant, the irradiated portion S of the fruit stalk 7 and If the distance from the reference position A is measured, the distance from the photographing means 9 to the irradiated portion S of the fruit stalk 7 can be calculated by using trigonometry.
Based on the distance between the irradiated portion S and the reference position A and the calculated distance from the photographing means 9 to the irradiated portion S of the peduncle 7, the control means 6 determines the irradiated portion of the peduncle 7 from the photographing means 9. Recognize the positional relationship with S.
Furthermore, since the collecting means 8 is fixed to the base member 5b together with the photographing means 9, the control means 6 controls the collecting means 8 and the fruit peduncle 7 based on the positional relationship between the photographing means 9 and the peduncle 7. The positional relationship with the irradiated portion S in 7 can also be recognized.

The control means 6 then controls the arm portion 5a of the robot 5 and the collecting means 8 to perform a harvesting step of harvesting the strawberries 1. FIG.
First, the control means 6 raises the collecting means 8 according to the recognized height of the irradiated portion S of the fruit stalk 7, and advances the collecting means 8 in the direction set in the entering direction setting step.
At this time, the control means 6 advances the collecting means 8 by a distance corresponding to the position of the fruit stalk 7 based on the positional relationship between the collecting means 8 recognized in the collecting position recognition step and the irradiated portion S of the fruit stalk 7 . Let
As a result, the peduncle 7 can be positioned between the grippers 11 of the picking means 8 , and then the control means 7 closes the gripper 11 to grip and cut the peduncle 7 . Harvest.
In this harvesting step, since the gripper 11 is moved based on the positional relationship between the harvesting means 8 and the fruit stalk 7 calculated in the harvesting position recognition step, the lengths of the gripping portion 11a and the cutting portion 11b of the gripper 11 are set short. However, the peduncle 7 can be positioned between them.
On the other hand, if the positional relationship between the picking means 8 and the peduncle 7 is not accurately recognized, even if the picking means 8 is moved in the direction set in the approach direction step, the grasping portion 11a and the cutting portion 11a will not be detected. It was necessary to set the portion 11b long and grip the peduncle 7 at a rough position.

The operation of the strawberry harvesting apparatus 2 having the above configuration will be described. is located on the side of the cultivation bed 3.
The photographing means 9 photographs a plurality of strawberries 1 from a position relatively distant from the cultivation bed 3, and the control means 6 recognizes the strawberries 1 to be harvested from the photographed images; 1, an approaching direction setting step for determining the direction in which the sampling means 8 is to approach.

The control means 6 causes the picking means 8, the photographing means 9 and the lighting means 10 to approach the strawberry 1 to be harvested from the approach direction set in the approach direction setting step, and performs the fruit pattern recognition step.
When the photographing means 9 photographs the image around the strawberry 1 to be harvested, the control means 6 recognizes the shape of the strawberry 1 and photographs so that the position of the center of gravity of the strawberry 1 coincides with the reference position A of the image. move the means 9;
Subsequently, the control means 6 causes the illumination means 10 to irradiate the line laser L obliquely downward, whereby part of the line laser L is irradiated to the fruit stalk 7 of the strawberry 1 to be harvested, and other lines are irradiated. The laser L passes behind the peduncle 7 .
As a result, the photographing means 9 photographs only the line laser L of the irradiated portion S irradiated to the fruit stem 7 of the strawberry 1, and the control means 6 determines that the irradiated portion S is the fruit stem 7 of the strawberry 1 to be harvested. recognize that there is
At this time, if the control means 6 cannot detect the peduncle 7 from the image, such as when another strawberry 1 is positioned behind the peduncle 7 or when there is a leaf in front of the peduncle 7, the harvesting and give a warning to that effect.

Next, the control means 6 performs a sampling position recognition step for recognizing the positional relationship between the photographing means 9 and the fruit stem 7 recognized in the fruit stem recognition step.
The control means 6 measures the distance between the irradiated portion S of the peduncle 7 and the reference position A from the image in which the peduncle 7 is recognized, and further determines the positions and imaging directions of the imaging means 9 and the illumination means 10. From the angle with the optical axis, the distance between the photographing means 9 and the irradiated portion S of the peduncle 7 is measured using trigonometry.
In addition, since the positional relationship between the photographing means 9 and the sampling means 8 is also registered in advance in the control means 6, the control means 6 can also calculate the positional relationship between the sampling means 8 and the irradiated portion S of the fruit stalk 7. can be done.

Thereafter, the control means 6 controls the arm portion 5a based on the calculated positional relationship to perform the harvesting step.
With the gripper 11 of the picking means 8 opened, the control means causes the gripper 11 to approach the peduncle 7 in the direction set in the approach direction setting step, and positions the peduncle 7 between the grippers 11. Let
When the peduncle 7 is positioned between the grippers 11 in this manner, the control means closes the gripper 11 and cuts the peduncle 7 with the cutting portion 11b while gripping the peduncle 7 with the gripping portion 11a. , harvest 1 strawberry.
Subsequently, the control means 6 moves the collecting means 8 to the tray 4b placed on the carriage 4 by the arm portion 5a, and stores the strawberries 1 in the tray 4b.

The collecting means 8 of this embodiment has a structure in which the gripper 11 is integrally provided with the gripping portion 11a and the cutting portion 11b. good.
Similarly, in the above-described embodiment, the sampling means 8, the photographing means 9, and the lighting means 10 are integrally fixed to the base member 5b provided at the tip of the arm portion 5a of the robot 5. may be moved individually.
At that time, if the positional relationship between these is recognized by the control means 6, the positional relationship between the photographing means 9 and the peduncle 7 of the strawberry 1 to be harvested can be recognized as in the above embodiment. The positional relationship between the means 5 and the peduncle 7 can also be recognized.
Further, the illumination means 10 irradiates the line laser L. As the line laser L, the tip of the laser irradiation means is reciprocated left and right or moved along a rectangular trajectory so that a line-shaped laser beam is emitted. Light may be formed.

Further, in the fruit pattern recognition step in the above embodiment, the fruit pattern 7 is not recognized by irradiating the line laser L to the fruit pattern 7, and the fruit pattern 7 is recognized by image recognition based on the method described in Patent Document 1. may be recognized, and then the collection position recognition step according to the above embodiment may be performed.
In the collecting position recognition step in this case, the lighting means 10 irradiates light obliquely downward, and the line laser L is irradiated to the fruit stalk 7 to obtain the irradiated portion S, so that the photographing means 9 and the fruit stalk 7 It is possible to recognize the positional relationship of
Further, the light irradiated to the fruit peduncle 7 in the collection position recognition step is not limited to the line laser L, and may be a spot light that forms the above-mentioned irradiated portion S on a part of the fruit peduncle 7. good.

1 Strawberry (fruit vegetable) 2 Strawberry harvesting device (fruit vegetable harvesting device)
3 cultivation bed 4 cart 5 robot 5a arm (moving means)
5b Base member 6 Control means 7 Fruit stem 8 Sampling means 9 Photographing means 10 Irradiation means L Line laser A Reference position S Irradiation part

Claims (3)

Picking means for grasping and picking fruit stalks of fruits and vegetables, moving means for moving the picking means, photographing means for photographing the fruits and vegetables, and the picking means by the moving means based on the image photographed by the photographing means. and a control means for causing the picking means to grasp the fruit stalk of the fruit vegetable by moving the
An irradiating means for irradiating a line of light formed in a lateral direction with respect to the fruit stalk is provided above the photographing means, and the optical axis of the irradiating means intersects the photographing direction of the photographing means from obliquely above . set up as
Harvesting fruits and vegetables, wherein the control means detects, from the image photographed by the photographing means, an irradiated portion of the fruit stalk irradiated with the line light, and recognizes the irradiated portion as the fruit stalk of the fruit vegetable. Device. Picking means for grasping and picking fruit stalks of fruits and vegetables, moving means for moving the picking means, photographing means for photographing the fruits and vegetables, and the picking means by the moving means based on the image photographed by the photographing means. and a control means for causing the picking means to grasp the fruit stalk of the fruit vegetable by moving the
An irradiating means for irradiating a line of light formed in a lateral direction with respect to the fruit stalk is provided above the photographing means, and the optical axis of the irradiating means intersects the photographing direction of the photographing means from obliquely above . set up as
The control means, in the image photographed by the photographing means, detects an irradiated portion where the line light is irradiated to the fruit peduncle, recognizes the distance between the irradiated portion and a reference position set in the image, A fruit and vegetable harvesting apparatus characterized by recognizing the positional relationship between the photographing means and the fruit stem based on the distance. 3. The apparatus according to claim 1, wherein said sampling means, said photographing means, and said irradiation means are provided on a base member provided on said moving means, and are integrally moved. vegetable harvesting equipment.

Images