JPH04311313A - Device of guiding harvester of fruits and vegetables - Google Patents

Abstract

PURPOSE:To facilitate harvest by reaping fruits having a smaller positional deviation between the central part of an image corresponding to color of fruits and a highlight zone when the difference in the highlight zones is <=a given value. CONSTITUTION:A central part W1 of a zone F1 corresponding to the color of fruits F and a central part W2 of a highlight zone F2 are obtained by a pickup image. When difference in brightness of the highlight zone is <=a given value, deviation (d) of position between the central part W1 and the central part W2 is obtained and fruits F having a little deviation (d) of position, being readily reaped, are preferentially harvested.

Description

[Detailed description of the invention]

0001

[Industrial Field of Application] The present invention includes a capturing unit that captures an object to be harvested, an imaging device that images the object to be harvested, and an image processing unit that performs image processing on the imaging information of the imaging device to identify a color corresponding to the color of the object to be harvested. an image processing means for determining a color region and a highlight region whose brightness is greater than a set value within the specific color region; and control for guiding the capture unit toward the harvesting target based on information from the image processing means. The control means is provided with a guidance device for a fruit and vegetable harvesting machine, wherein the control means is configured to give priority to a harvest target having a high brightness in the highlight area.

0002

2. Description of the Related Art Conventionally, when the difference in brightness between highlight areas of a plurality of harvesting targets is smaller than a predetermined value, priority is given to the harvesting target with a larger highlight area.

0003

[Problems to be Solved by the Invention] Generally, the area of the highlight region of a large harvest target is large. However, it is not necessarily the case that the larger the highlight area is, the easier it is to harvest. For example, when harvesting fruits and vegetables that are partially shielded by leaves, etc., the leaves may get in the way, making it impossible to properly absorb the fruits and vegetables and cut the stalks, making it difficult to harvest them properly. Therefore, the above-mentioned conventional techniques cannot improve the efficiency of harvesting fruits and vegetables. An object of the present invention is to eliminate the above-mentioned conventional drawbacks and provide a guidance device for a fruit and vegetable harvesting machine with high working efficiency.

0004

[Means for Solving the Problems] In order to achieve this object, the characteristic structure of the guidance device for the fruit and vegetable harvesting machine according to the present invention is as follows:
The image processing means is configured to determine a positional shift of the highlight area with respect to the center or center of gravity of the specific color area, and the control means determines when the difference in brightness of the highlight area is smaller than a predetermined value. The present invention is configured to give priority to harvesting targets with small positional deviations.

[0005]

[Operation] If the harvested object can be imaged without being obstructed by leaves or the like, the highlight area is considered to be in the center of the specific color area or near the center of gravity. However, if a portion of the area is blocked by leaves or the like, the highlight area will be shifted in the direction of the leaves or the like from the center or center of gravity of the specific color area. Therefore, by prioritizing the harvesting targets with small positional deviations, fruits and vegetables that are easy to harvest are preferentially harvested.

[0006]

[Effects of the Invention] By preferentially harvesting fruits and vegetables that are easy to harvest, the harvesting operation can proceed smoothly. Furthermore, when one harvest target is harvested, the situation such as leaves that shield other harvest targets that remain unharvested may change, making harvesting easier. That is, it can contribute to increasing the working efficiency of the guidance device of the fruit and vegetable harvesting machine.

[0007]

Embodiments Hereinafter, embodiments in which the present invention is applied to a guidance device for a fruit harvesting machine will be described with reference to the drawings. As shown in FIG. 7, a boom 2 is attached to a vehicle body V equipped with a pair of left and right running wheels 1 at the front and rear so that it can be driven up and down and rotated, and an auxiliary boom 3 is attached to the tip of the boom 2 in a horizontal direction. A working manipulator 4 is attached to the tip of the auxiliary boom 3 so as to be swingable and equipped with a hand H for harvesting fruit. In addition, in the figure,
5 is a hydraulic cylinder for lifting and lowering the boom, 6 is a hydraulic cylinder for swinging the boom, and 7 is a hydraulic cylinder for swinging the auxiliary boom.

The working manipulator 4 is composed of a multi-jointed telescopic arm 8 and a hand H as a catching section attached to the tip of the telescopic arm 8. The telescoping arm 8 is rotated around the vertical axis Y by the electric motor 9a, swung around the horizontal axis X by the electric motor 9b, and extended and retracted by the electric motor 9c. It is configured as follows.

To explain the hand H, as shown in FIGS. 8 and 9, there is a vacuum pad 10 at its tip for sucking the fruit F to be harvested, and a suction port 10a of the vacuum pad 10. A trapping part case 12 for covering the adsorbed fruit F, and the trapping part case 12
A handle cutting device 13 is provided which cuts the handle part of the fruit F taken into the fruit while holding it in the vertical direction.

To further explain the vacuum pad 10, the vacuum pad 10 is formed into a bellows shape, and is attached to the ventilation pipe 11 serving as the main body so that the direction can be changed freely and the vacuum pad 10 is biased to return forward. ing. Further, three air cylinders 21 connected to the suction port 10a at the tip via wires 22, and three proximity switches S4 are attached to the ventilation pipe 11. The air cylinders 21 and the proximity switches S4 are alternately arranged at intervals of 60 degrees on the same circumference. Also, the suction port 10a
protrusions 10b at positions facing the proximity switches S4, respectively.
are provided so as to individually detect the backward movement of the suction port 10a with respect to the ventilation pipe 11. By the way, by moving the three air cylinders 21 in and out at the same time, they can be moved in and out of the ventilation pipe 11 with the suction port 10a facing forward. Also, by retiring one or two air cylinders 21 and extending the other air cylinders 21, the ventilation pipe 1 of the vacuum pad 10 can be removed.
The direction relative to 1 can be changed. In the figure,
S1 is a color image sensor serving as an image capturing means for capturing an image of the fruit F located in the direction in which the working hand H is facing;
S2 is a reflective proximity sensor that uses infrared light to detect when the work hand H approaches the fruit F to be worked within a set distance, and both sensors S1 and S2 detect when the work hand H approaches the fruit F to be worked within a set distance. It is provided inside the vacuum pad 10 in order to prevent this from happening. Further, 15 is a lighting device for illuminating the front side of the working hand H with a set light amount in synchronization with the imaging processing of the image sensor S1, and generally a so-called strobe light emitting device is used. .

The catcher case 12 includes a proximal end portion 12a that is externally supported by the ventilation pipe 11 so as to be slidable in the front-back direction of the hand, and an upper cover 12 that is fixed to the upper portion of the distal end side of the proximal end portion 12a.
b, and a lower cover 12c that is also fixed to the lower part of the tip side.
It consists of Then, until the vacuum pad 10 adsorbs the fruit F, the hand is retired to the rear side so as not to get in the way of the approach, and after the vacuum pad 10 has adsorbed the fruit F, it is set so as to cover the adsorbed fruit F. It is designed to protrude toward the front of the hand. A shutter 25 is provided at the lower rear side of the lower cover 12c, and can be opened and closed by an air cylinder 23 for opening and closing. Further, the shutter 25 is provided with a lever 24 made of resin that can freely swing around the horizontal axis, and a proximity switch S5 that detects the swing of the lever 24. Only the lower part of the lever 24 is made of metal to urge it back to the upright position, and when the fruit F whose handle has been cut falls onto the shutter 25, it is swung to the laid down position. The proximity switch S
5 is turned on when the lever 24 is in the prostrate position. With that,
Cutting of the stalk of fruit F can be detected.

The handle cutting device 13 includes a clipper-shaped blade 13a that cuts the handle of the fruit F, and a handle support member 13b that presses and supports the handle of the fruit F from below on the blade 13a.

Although not described in detail, the hand H is equipped with various pneumatic actuators for harvesting operation, such as an actuator for moving the trap case 12 in and out, and an actuator for driving the handle cutting device 13. , various sensors such as a negative pressure sensor S3 that detects that the fruit F is adsorbed to the vacuum pad 10, a sensor that detects the exit/removal position of the trap case 12, and a sensor that detects the driving state of the handle cutting device 13. A category will be established.

[0014] Adding an explanation to the fruit harvesting procedure,
With the catching part case 12 retired to the rear side of the hand and the suction port 10a of the vacuum pad 10 protruding forward beyond the catching part case 12, the working hand H is positioned at a preset approach start position, In addition, the image sensor S is directed toward the direction in which the fruit F to be harvested is located.
Image processing is performed in step 1. Then, the approach order for the fruit F is determined based on information obtained by image processing the captured image information, and the approach direction of the working hand H is determined. After determining the approach direction, each electric motor 9a, 9b, 9c is operated so that the work hand H moves linearly toward the determined approach direction, and the proximity sensor S2 detects the fruit F. Accordingly, the vacuum pad 10 is operated to attract the fruit F to be harvested, and the catching part case 12 is protruded to take the fruit F into its inside, and then the handle is cut and the fruit F is harvested. become. However, after the proximity sensor S2 is activated, if the negative pressure sensor S3 is not activated while the hand H is moved forward by the set distance that the proximity sensor S2 is activated, it is determined that the harvest has failed, and the set distance is Based on the detection information of the proximity switch S4, the position of the hand H is corrected by a predetermined distance in the direction in which the vacuum pad 10 has changed its direction from the forward direction, and the hand H is reapproached from the position where the hand H is moved back toward the front side. It has become. by the way,
The direction of the vacuum pad 10 is, for example, the proximity switch S.
When only one of the four proximity switches S4 is turned on, it is determined that the direction has been changed, and when two of them are turned on, it is determined that the direction has been changed to a direction intermediate between the two proximity switches S4.

To explain the determination of the approach order, as shown in FIGS. 6(A) and 6(B), there is a specific color area F1 corresponding to the color of the fruit F, and the brightness within the specific color area F1 is determined. A highlight area F2 larger than the set value,
The center of gravity W1 of the specific color area F1 and the center of gravity W2 of the highlight area F2 are determined. Then, basically, based on the brightness of the highlight area F2, the fruit F with the highest brightness is selected.
Determine the approach order by giving priority to However, when the difference in brightness of the highlight area F2 is smaller than a predetermined value,
The positional deviation d between the center of gravity W1 of the specific color area F1 and the center of gravity W2 of the highlight area F2 is determined, and the approach order is determined by giving priority to the fruit F with the smaller positional deviation d.

Next, a control configuration for determining the approach order of the fruit F and guiding the hand H toward the fruit F based on the detected information such as image information captured by the image sensor S1 will be explained. As shown in Figure 1,
A control device 16 using a microcomputer is provided,
The control device 16 includes the sensors S1, S2, S3,
S4 and S5 are connected. Then, the control device 16 controls the air cylinder 21, each of the hydraulic cylinders 5 and 6, based on preset and stored information and various input information.
7 and each electric motor 9a, 9b, 9c. That is, the image processing means 100 and the control means 101 are configured using the control device 16.

Next, the operation of the control device 16 will be explained based on the flowcharts shown in FIGS. 2 to 5. First, the hand H is set at a preset approach start position, and imaging processing is performed with all air cylinders 21 retired.
The captured image information is subjected to image processing. In other words, n in the image
The specific color area F1, the highlight area F2, the center of gravity W1 of the specific color area F1, and the center of gravity W2 of the highlight area F2 of each fruit F are determined. Further, the number n of fruits F is determined based on the number of specific color regions F1 and highlight regions F2, and after assigning a number from 1 to n to each fruit F,
As described above, the approach order is determined by comparing the brightness or positional deviation d of each highlight area F2. Then, the center of gravity W2 of the approaching fruit F
Move hand H in the direction where is located.
The approach direction of the hand H is determined and the telescoping arm 8 is extended. At this time, all the air cylinders 21 are operated in a protruding manner. After the extension of the telescopic arm 8 is started, based on the information of the proximity sensor S2, it is determined whether or not the hand H has approached the target fruit F to be harvested by the predetermined distance, As the fruit approaches, the extension speed of the telescopic arm 8 is switched to a low speed, the suction operation of the vacuum pad 10 is started, and the hand H is further moved toward the fruit side until the negative pressure sensor S3 is turned on. When the negative pressure sensor S3 turns on and detects fruit adsorption, the telescopic arm 8 is stopped and the harvesting operation is started. However, if the negative pressure sensor S3 does not operate after the proximity sensor S2 is turned ON and the negative pressure sensor S3 does not operate while moving the set distance, it is determined that the harvest has failed. At this time, the proximity switch S
4 is turned on, the hand H is once returned to the position where the proximity sensor S2 was turned on, the position of the hand H is corrected, and the hand H is approached again as described above. Both proximity switches S4 are ON
If it is not working, it is judged that it is impossible to harvest and the hand
Return to the approach start position. When the fruit can be harvested by the first approach or the second approach, return the working hand H to the approach starting position while holding the fruit F with the cut stalk inside the trap case 12,
Fruit F will be discharged. After discharging the harvested fruits F, it is determined whether or not the work is completed based on the number n, and if the work is not completed, the work hand H is reset to the approach start position and image processing is performed. ,and,
The approach order for the remaining fruits F is determined, and each process of guiding the working hand H toward the determined approach direction is repeated.

[Another Embodiment] In the above embodiment, the present invention was applied to a guidance device for a fruit harvesting working machine. It can be applied to a commercial work machine, and the specific configuration of each part, such as the specific configuration of the hand H, can be changed in various ways. Incidentally, although reference numerals are written in the claims section for convenient comparison with the drawings, the present invention is not limited to the structure shown in the accompanying drawings.

[Brief explanation of drawings]

[Figure 1] Block diagram of control configuration

[Figure 2] Flowchart of control operation

[Figure 3] Flowchart of control operation

[Figure 4] Flowchart of control operation

[Figure 5] Flowchart of control operation

[Figure 6] Explanatory diagram of image processing

[Figure 7] Schematic side view of harvester

[Figure 8] Side view of the main parts of the hand

[Figure 9] Front view of main parts of hand

[Explanation of symbols]

100 Image processing means 101 Control means S1 Imaging means H Capture part F1 Specific color area F2 Highlight area W1 Central part or center of gravity d　　　　　Position shift

Claims (1)

[Claims] [Claim 1] A capturing section (H) that captures a harvest target;
An imaging means (S1) for taking an image of the harvesting target, and image processing of the imaging information of the imaging means (S1) to produce a specific color area (F1) corresponding to the color of the harvesting target and its specific color area (F1).
1), an image processing means (100) for determining a highlight region (F2) whose brightness is greater than a set value, and an image processing means (100) for determining a highlight region (F2) in
H) towards the harvest target (101);
), and the control means (101) is a guidance device for a fruit and vegetable harvesting machine configured to give priority to a harvest target whose brightness in the highlight area (F2) is large; The image processing means (100) processes the specific color area (F
1), and the control means (101) is configured to determine the positional deviation (d) of the highlight area (F2) with respect to the central part or center of gravity (W1) of the highlight area (F2).
) is smaller than a predetermined value, the guidance device for a fruit and vegetable harvesting machine is configured to give priority to the harvesting target with the smaller positional deviation (d).