JP3757279B2 - Method and apparatus for selective harvesting of fruit vegetables - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention selects fruits of suitable harvest time from among a mixture of fruits such as eggplant, cucumber, peppers, and strawberries that are suitable for harvest and before harvest, and approaches the fruits to be harvested without touching the fruits. The present invention relates to a method and an apparatus for automatically harvesting efficiently in a cultivation facility by cutting and harvesting a handle.
[0002]
[Prior art]
Conventionally, eggplants and cucumbers are harvested manually, and the fruits at the best time to harvest are selected and harvested from the mixture of the fruits at the right time and the fruits before the right time.
In recent years, methods and apparatuses for automatic harvesting of cucumbers and the like have been proposed (see, for example, Patent Document 1 and Patent Document 2). In this prior art, since it is based on holding and harvesting a fruit part, there exists a possibility of damaging a fruit. Moreover, since the cutting position is determined in a state where the tip of the manipulator is close to the fruit, it takes time for the approach.
[0003]
In addition, a method and apparatus relating to the automatic harvesting of eggplant has been proposed (see, for example, Patent Document 3). In this prior art, as in the case of automatic harvesting such as the cucumber, it is a mechanism for gripping and picking the fruit part, and it is determined whether or not harvesting is appropriate in a state where the end effector is close to the distance in contact with the fruit. Harvest man-hours are required. Furthermore, the harvesting method combined with the movement mechanism under natural light in the facility is not shown. Therefore, a problem remains in using the harvesting apparatus efficiently in the cultivation facility.
[0004]
[Patent Document 1]
Japanese Patent No. 3018581 [Patent Document 2]
JP-A-8-56459 [Patent Document 3]
Japanese Patent No. 3054706 gazette
[Problems to be solved by the invention]
Therefore, automatic harvesting of fruits and vegetables is not realized, and the harvesting work is still carried out manually and forced to work for a long time. In addition, harvesting is performed early in the morning to prevent quality degradation, and the fruit is carefully harvested so as not to damage the skin.
The present invention was made in order to solve the above problems and problems, and reduces the determination process of harvest suitability while traveling between furrows in a facility where fruit vegetables such as eggplant are grown. It is an object of the present invention to provide a method and apparatus for selectively harvesting fruits at an appropriate harvest time without damage.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides
A. Analyzing the whole image of fruit and vegetables taken with a camera attached to the tip of the manipulator, and calculating the direction of the fruit in the screen, the manipulator capable of vertical and horizontal movement, vertical and horizontal turning is controlled, After approaching the manipulator tip in the direction of the fruit, the image data re-captured by the camera and the distance data from the manipulator tip to the fruit are fused to determine the size of the fruit and the position of the fruit pattern. The fruit handle is cut by bringing it close to the fruit handle, and the fruit handle is gripped by the lower blade for holding the fruit handle attached to the lower side of the fruit scissors and harvested without touching the fruit part. Selective harvesting of fruits and vegetables characterized by selectively harvesting fruits at the appropriate harvest time in the cultivation facility by repeating and sequentially moving the fruits on both sides of the vehicle It is the law.
[0007]
B. A manipulator that is installed at the center of the left and right of the traveling vehicle so that the tip reaches the fruit of the left and right cocoons, can move up and down, left and right, and back and forth, vertical and horizontal, a camera attached to the tip of the manipulator, and the distance to the fruit Based on a distance sensor to measure, an image device that analyzes a captured image and calculates a fruit position and a fruit handle position, a computer that determines a movement position of the manipulator based on the calculation result, and a calculation result of the computer, A controller for driving the manipulator, and a fruit collection scissors having a lower blade for holding a fruit handle that is opened and closed by a digital signal from the computer, and the image device analyzes an entire image of fruit vegetables taken by the camera, From the computer result of calculating the direction of the fruit in the screen, the controller manipulates the manipulator. Then, the tip of the manipulator is made to approach toward the fruit, and the computer fuses the image data taken again by the camera and the distance data from the tip of the manipulator to the fruit to determine the size of the fruit and Selective harvesting of fruit vegetables characterized by performing position detection, cutting the fruit handle by bringing the fruiting scissors closer to the fruit pattern, and outputting a signal for gripping the fruit pattern by the lower blade for holding the fruit pattern Device.
[0008]
[Action]
With the above means and configuration, when automatically harvesting fruits and vegetables cultivated in the facility, only images in a certain area are captured with a camera attached to the tip of the manipulator, and analyzed and detected by a computer. The manipulator tip is approached in the direction of the fruit, and the size of the fruit is estimated by fusing the image data and the distance data taken again with the camera. Fruits can be judged automatically. In addition, the pattern position, that is, the position to insert the harvesting scissors is determined by this sensing, so that the harvesting process can be reduced. In addition, after the fruit scissors cut the fruit handle, the end effector part contacts the fruit skin to convey the fruit to the harvesting box while holding the fruit by the lower knife for holding the fruit handle that operates in conjunction with the opening and closing of the fruit scissors. It is possible to prevent damage to the fruit without doing so.
In addition, since the traveling vehicle can move to any position in the cultivation facility, it is possible to determine the harvest suitability and the harvesting operation by changing the position of the traveling vehicle even for the fruit hidden behind the foliage and not detectable from the front position. This will enable efficient harvesting operations.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention adapted to eggplant harvest will be described in detail with reference to the drawings.
As shown in FIGS. 1 and 2, the fruit and vegetables selective harvesting apparatus 1 according to the present invention includes a traveling vehicle 2, a control unit 3, a visual unit 4, a manipulator unit 5, an end effector unit 6, and the like.
The traveling vehicle 2 drives the traveling vehicle motor 2 a with a power source from a battery or the like by a digital signal from the control unit 3, and moves on the piping pipe 7 laid in the facility. The travel distance is detected by the traveling vehicle rotary encoder 2b, and the traveling vehicle 2 can be stopped at an arbitrary position. When there is an obstacle in the traveling track, the limit switch A2c or the limit switch B2d is activated, and the traveling vehicle 2 is automatically stopped.
[0010]
The control unit 3 includes a computer 3 a, an AD / DA conversion board 3 b that converts the output value of the ultrasonic distance sensor 6 a, a digital I / O board 3 c that controls the fruit scissors 6 b and the traveling vehicle 2, and the traveling vehicle 2. And an encoder counter board 3d for detecting the movement distance. The computer 3a is installed behind the traveling vehicle 2, and an AD / DA conversion board 3b, a digital I / O board 3c, and an encoder counter board 3d are inserted therein.
[0011]
The visual unit 4 employs a CCD color camera 4a and is attached to the tip of the manipulator 5a. Further, an image processing board 4b is provided.
The manipulator unit 5 includes a manipulator controller 5b that performs communication with the computer 3a, and a manipulator 6a that operates based on the communication information and performs a harvesting operation. The manipulator controller 5b is installed behind the traveling vehicle 2 and adjacent to the computer 3a. However, the computer 3a and the manipulator controller 5b may be stored in the lower part of the manipulator 5a. The manipulator 5a is installed at the center in the left-right direction of the traveling vehicle 2 so that fruits on both sides can be harvested.
[0012]
As shown in FIG. 3, the end effector section 6 includes an electric open / close hand 6c that opens and closes by a signal from the computer 3a, a fruit scissor 6b that cuts an eggplant fruit pattern by a reciprocating motion of the electric open / close hand 6c, and a sampling. A fruit handle clamping lower blade 6d that is attached to the lower side of the fruit scissors 6b and opens / closes with the operation of the fruit scissors 6b to clamp the fruit handle, and when the manipulator 5a approaches the fruit, the distance to the fruit is measured. The ultrasonic distance sensor 6a is attached to the tip of the manipulator 5a.
[0013]
Hereinafter, the operation of the fruit and vegetables selective harvesting apparatus 1 having the above configuration will be described.
As shown in FIGS. 4 and 5, the traveling vehicle 2 starts the scanning origin, scans the eggplant image at the positions of the lower, middle, and upper left and right sides of the manipulator 5a, and performs harvesting processing. When the harvest of is finished, it moves by the distance between the stocks, and stops before the next eggplant stock. This automatic harvesting operation is repeated, and when the harvesting of the target range is completed, the scanning origin is restored.
[0014]
As shown in FIG. 6, the selective harvesting apparatus 1 moves in the middle of the passage between the ridges where eggplants are cultivated, and reaches the end of the manipulator 5 a to the fruit that has reached the branch as a result of extending from the node of the main stem inclined toward the passage. Can be made. Further, the manipulator 5a is installed at the center in the left-right direction of the traveling vehicle 2, and the pipe pipe 7 is laid at the center of the passage, whereby the fruits on both sides can be harvested by one traveling. When harvesting from the lower stage or the middle stage, it is easy to detect the leaf fruit in the elevation direction of the camera 4a. Furthermore, even if the fruit cannot be detected from the front of the eggplant stock, the viewpoint of the camera 4a can be moved in the front-rear direction of the traveling vehicle 2 by the traveling function, so that the fruit detection rate is improved.
[0015]
The approach to the fruit and the fruiting operation after the traveling vehicle 2 stops in front of the stock of eggplant and the tip of the manipulator 5a is moved to the initial position of the lower, middle or upper stage will be described in detail with reference to FIG. An eggplant image is taken by the CCD color camera 4a at the initial position and image processing is performed. This image processing method will be described later. The detection result as shown in FIG. 8 is obtained by image processing, and the fruit direction is determined based on the displacement from the center position of the screen. Then, the detected fruit is approached in the order from the upper left to the lower right, and the harvesting process is performed. The harvesting order may be the order close to the camera 4a, that is, the order close to the center of the screen. The tip of the manipulator 5a is turned and advanced in the direction of the first fruit, and the distance to the fruit is measured by the ultrasonic distance sensor 6a.
[0016]
Further, by repeating image processing, which will be described later, and turning and forward / backward movement of the tip of the manipulator 5a, the fruit approaches until it fills the screen. Image processing is performed again at this position, and suitability for harvesting is determined based on the fruit length. In the case of strawberries and the like, the suitability for harvesting is determined by the redness of the fruit portion. If the fruit is suitable, the tip of the manipulator 5a is turned and advanced in the direction of the fruit base, and the fruit handle is sandwiched between the fruit scissors 6b as shown in FIG. The electric open / close hand 6c is closed by a digital signal from the computer 3a, and the fruit pattern is cut by the fruit collection scissors 6b. Simultaneously with this operation, the lower blade 6d for holding the fruit pattern closes, and even if the fruit pattern is cut, it is held by the lower blade 6d, so that the fruit does not fall. Since the eggplant handle is strong, it escapes backward when cutting, but as shown in FIG. 10, the fruiting scissors 6b are closed and the fulcrum is moved forward, and the tip of the manipulator 5a is controlled forward. You can cut and hold the escape. The last harvested fruit is put in the harvest box 8. This series of operations is continued until there is no fruit detected at the initial position. In this harvesting method, the fruit quality can be improved because it does not come into contact with the fruit in the process from the harvesting to the transport to the harvesting box 8. Especially for strawberries and the like, the quality is liable to deteriorate due to the contact of the fruits, so that the merit of holding the fruit pattern and carrying it is great.
[0017]
Next, the initial position of the tip of the manipulator 5a and the image processing method during the approach to the fruit will be described in detail with reference to FIG.
The B image (blue component) is subtracted from the G image (green component) of the color image photographed by the CCD color camera 4a and binarized to extract a portion other than green. A dark part other than the background is extracted by binarizing the G image, and an image of a logical product of the part other than green and the part other than the background is obtained. The obtained image is subjected to expansion, contraction, and hole filling processing to extract a black-purple fruit portion including a fruit pattern as shown in FIG. In the case of strawberry or the like, the fruit portion is extracted based on the ratio of the R image (red component) and the G image (green component). Then, by performing contraction and expansion processing in the horizontal direction by the number of pixels on the image corresponding to the fruit pattern diameter, the part above the fruit base is removed and only the fruit part is extracted. Further, by performing contraction and expansion processing in the vertical direction by the number of pixels corresponding to the minimum fruit length of the fruit suitable for harvest on the image, the short fruit before the harvest proper period is removed.
[0018]
Next, when the tip of the manipulator 5a is at the initial position, there may be several fruits suitable for harvest on the screen, so the number of fruits, their respective areas, and the position of the center of gravity are calculated. On the other hand, when the tip of the manipulator 5a approaches the fruit to be harvested, the calculation is performed for only the fruit of the maximum area, and the center of gravity position, the maximum length, the maximum width, and the fruit base position are calculated. In the case of eggplant and cucumber, the determination of harvest suitability is performed based on the fruit length estimated from the maximum length shown in FIG. 12, but for round eggplant varieties and peppers, it is also possible to determine the combination of fruit length and fruit width. is there.
[0019]
The result of having performed the operation test of the fruit and vegetables selective harvesting apparatus 1 according to the present invention will be described. The image scanning test was performed twice and the harvesting operation test was performed four times for the area of the heel direction distance of 4.0 m, height of 1.7 m, and 20 strains on both sides in the facility.
FIG. 13 shows the result of image processing for fruit detection performed on the image taken at the initial position. Even in a facility under natural light, 59.6 to 83.3% (total average of 69.3%) of fruits visible from the passage side can be automatically detected, and fruits of adjacent strains are also 25.0 to 63.63. Detection was possible at 9% (overall average 40.9%). However, even when viewed from the front and left and right directions, the fruit was not detected when the fruit was hidden in the foliage.
[0020]
The test result by the selective harvesting apparatus 1 is shown in FIG. The success rate of harvesting was affected by the way of pruning the leaves and was 12.8-46.7% (overall average 29.1%). The erroneous harvest of the fruits before the appropriate harvest time was 0.0 to 8.0% (overall average 3.5%). In addition, there was no damage to the pericarp on fruits that were successfully harvested. Since the diameter of the cut fruit pattern portion is smaller than the gap between the fruit pattern sandwiching lower blades 6d, 0.0 to 10.6% (total average 7.6%) of fruits fell in the transporting process to the harvesting box 8. . The main cause of the harvest error was a fruit detection error due to image processing resulting from hiding the fruit in the foliage.
[0021]
Regarding the operation time of the selective harvesting apparatus 1, the time required for image scanning of the set area was 312 to 319 seconds. Further, the harvesting processing time was 965 to 3157 seconds, and the relationship between the number of fruits at the appropriate harvesting time and the harvesting processing time increased proportionally with the increase in the number of fruits as shown in FIG. The harvesting work efficiency including the return to origin was 4.56 to 14.92 m / h (overall average 6.98 m / h). The number of fruits harvested per hour was 9.1 to 26.2 fruits / h (overall average 20.5 fruits / h). In addition, the operation time required only for the harvesting processing of the fruit which was successfully harvested was 43.2 seconds / fruit on average.
[0022]
Next, about the harvest state of the fruit, as shown in FIG. 16, it cut | disconnected 25.6-28.6 mm upper side from the fruit base, and the cut diameter was 5.9-6.3 mm. Although the indentation at the time of pinching remained in the fruit handle, the fruit skin was not damaged because the fruit was not grasped in the fruit collection and transportation process. As shown in FIG. 17, the cut position of the fruit that fell midway varied from 16.0 to 34.0 mm above the base, and the diameter of the cut surface was 3.5 to 4.0 mm.
[0023]
【The invention's effect】
As described above, according to the fruit vegetable harvesting method and apparatus of the present invention, the following effects can be achieved by having the means and configuration according to claims 1 and 2.
While running through the furrows of fruit and vegetables grown in a facility under natural light where the amount of solar radiation changes, the fruit on both sides is detected, the direction of the fruit and the suitability of harvest are determined, and the position of the fruit pattern is detected. Can be cut and sandwiched and stored in a harvesting box, providing automatic and efficient harvesting technology.
Further, since the fruit pattern is sandwiched simultaneously with the cutting of the fruit pattern, it can be gripped and transported without contacting the fruit, and the fruit quality is not deteriorated without damaging the fruit skin.
[Brief description of the drawings]
FIG. 1 is an overall view of a selective harvesting apparatus.
FIG. 2 is a block diagram of a selective harvesting apparatus.
FIG. 3 is a side view (a), a front view (b), and a bottom view (c) of an end effector section.
FIG. 4 is a process chart of a method for harvesting fruit on both sides of the furrow.
FIG. 5 is a flowchart of furrow running and harvesting processing.
FIG. 6 is a conceptual diagram of traveling between furrows.
FIG. 7 is a flowchart of a method for approaching a fruit and a fruit harvesting operation.
FIG. 8 is an explanatory diagram for performing fruit detection image processing and fruit direction estimation;
FIG. 9 shows a fruit pattern cutting operation and a fruit pattern clamping operation.
FIG. 10 is an open / close view of the fruit scissors, showing a closed state (a) and an open state (b).
FIG. 11 is a flowchart of image processing for fruit detection.
FIG. 12 is an explanatory diagram showing a method for detecting a fruit base and determining whether or not to harvest.
FIG. 13 is a graph showing fruit detection results by image processing.
FIG. 14 is a graph showing the result of harvesting eggplant fruits in the facility.
FIG. 15 is a graph showing the relationship between the number of fruits in a suitable harvest period and harvest processing time.
FIG. 16 is a graph showing the fruit pattern cutting position and fruit pattern diameter of the cut part of successfully harvested fruits.
FIG. 17 is a graph showing the fruit pattern cutting position and fruit pattern diameter of a cut part of a fruit falling midway.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Fruit and vegetable selection harvesting device 2 Traveling vehicle 2a Traveling vehicle motor 2b Traveling vehicle rotary encoder 2c Limit switch A 2d Limit switch B
DESCRIPTION OF SYMBOLS 3 Control part 3a Computer 3b AD / DA conversion board 3c Digital I / O board 3d Encoder counter board 4 Visual part 4a CCD color camera 4b Image processing board 5 Manipulator part 5a Manipulator 5b Manipulator controller 6 End effector part 6a Ultrasonic distance sensor 6b Harvesting scissors 6c Electric open / close hand 6d Lower blade 7 for gripping fruit handle Piping pipe 8 Harvesting box

Claims (2)

Analyzing the whole image of fruit and vegetables taken with a camera attached to the tip of the manipulator, and calculating the direction of the fruit in the screen, the manipulator capable of vertical and horizontal movement, vertical and horizontal turning is controlled, After approaching the manipulator tip in the direction of the fruit, the image data re-captured by the camera and the distance data from the manipulator tip to the fruit are fused to determine the size of the fruit and the position of the fruit pattern. The fruit handle is cut by bringing it close to the fruit handle, and the fruit handle is gripped by the lower blade for holding the fruit handle attached to the lower side of the fruit scissors and harvested without touching the fruit part. Selective harvesting of fruits and vegetables characterized by selectively harvesting fruits at the appropriate harvest time in the cultivation facility by repeating and sequentially moving the fruits on both sides of the vehicle Law. A manipulator installed at the center of the left and right of the traveling vehicle so that the tip reaches the fruits of the left and right cocoons, and capable of vertical and horizontal turning, vertical and horizontal turning,
A camera attached to the tip of the manipulator;
A distance sensor that measures the distance to the fruit;
An image device that analyzes the captured image and calculates the fruit position and the fruit pattern position;
A computer for determining a movement position of the manipulator based on the result of the calculation;
A controller for driving the manipulator based on a calculation result of the computer;
A fruit scissors having a lower blade for pinching a fruit handle that is opened and closed by a digital signal from the computer,
The whole image of fruit and vegetables taken by the camera is analyzed by the imaging device, and the manipulator is controlled by the controller from the result of the computer that calculates the direction of the fruit in the screen, and the tip of the manipulator is moved in the fruit direction. Close to
The computer
The image data taken again by the camera and the distance data from the tip of the manipulator to the fruit are fused to determine the size of the fruit and the position of the fruit pattern, and the fruiting scissors are brought close to the fruit pattern to remove the fruit pattern. A selective harvesting apparatus for fruit vegetables, which cuts and outputs a signal for gripping the fruit pattern by the lower blade for sandwiching the fruit pattern .

Images