JP3506779B2 - Fruit and vegetable harvest target detection system - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup means for picking up an image of a predetermined range including a harvested target portion of fruit and vegetables, and a harvested target position detection for detecting the position of the harvested target portion based on image pickup information of the image pickup means. And a means for detecting a target portion for harvesting fruits and vegetables provided with means.

[0002]

2. Description of the Related Art In such an apparatus for detecting a target portion for harvesting fruits and vegetables, a predetermined range including a target portion for harvesting fruit or fruits, which consists of vegetables such as cucumber and fruits such as mandarin orange, that is, a fruit portion is imaged by a color CCD camera or the like. The image is picked up by the means, and in the picked-up image, for example, a region that matches the shape and color of the fruit portion stored in advance is recognized as the fruit portion region,
The position of the area was detected as the position of the fruit part. Incidentally, based on the position detection information of the fruit portion, for example, a harvesting hand equipped with a fruit gripping portion and a cutter for cutting the handle thereof is guided toward the fruit portion, and the gripping portion grips the fruit portion. In this state, the handle is cut and the fruit is harvested.

[0003]

However, in the above prior art, when a part or all of the harvest target portion (fruit portion) is hidden by another object such as a leaf, a captured image of the entire harvest target portion is obtained. It was not possible to obtain it, and it was difficult to recognize the harvesting target portion based on the characteristics such as the shape and detect its position. Also, for example, when the target part to be harvested is red and the color is different from the green color such as leaves such as tomatoes, when recognizing the target part to be harvested based on the color difference, part of the target part to be harvested is a leaf or the like. Even if it is hidden by, it is possible to recognize the area of the harvest target part and perform position detection based on the area information, but in that case, the position displaced from the original position of the harvest target part is There was a risk of false detection as a position.

The present invention has been made in view of the above circumstances, and an object thereof is to obtain a part or all of a harvested part (fruit part) from another object such as a leaf in an image pickup screen of an image pickup means. Even if it is hidden by, the position of the harvested part can be detected reliably and with high accuracy, and the drawbacks of the above-mentioned conventional techniques are eliminated.

[0005]

A first characteristic configuration of the apparatus for detecting a target portion for harvesting fruit and vegetables according to the present invention is that another object having a lighter weight than the target portion for harvesting is present within the image capturing range of the image capturing means. An air blower is provided to move the air blower away from the object to be harvested, and the object to be harvested position detection means is based on imaged information during operation of the air blower, or
It is configured to detect the position of the harvesting target portion based on the imaged information when the blower is operating and the imaged information when the blower is not operating. Also, the first feature
The configuration is such that the imaging unit and the air blowing unit are
It is installed on the harvesting hand that is operated toward the elephant
There is a point. Further, the first characteristic configuration is the above-mentioned harvesting ha
And a blower for grasping a harvested part provided in a wind
The air supply means for supplying operating air to the stage is
Supply of operating air to the gripping part and
It is configured to be switchable between the supply of operating air
There is a point.

A second characteristic configuration is the same as the first characteristic configuration, and the blower means includes the harvesting target portion in a blown air state in which the other object is moved but the harvesting target portion is not moved. The point is that the air is blown toward a predetermined range.

[0007]

[0008]

[0009]

According to the first characteristic construction of the present invention, when the air blowing means does not operate, other substances such as leaves having a lighter weight than the harvested portion of the fruit and vegetables are present within the image pickup range of the image pickup means. Although it is in a state where it cannot be distinguished because it overlaps with the target part, when the air blowing means operates, other objects such as the leaves move with respect to the target part and the target part existing within the imaging range of the imaging means Separate. Then, based on the imaged information when the blowing unit is operated in a state where the harvest target portion and other objects such as leaves are separated in the imaging screen, the harvest target portion is identified from the shape and color characteristics of the harvest target portion, for example. The position of the harvested part is detected, or based on the both imaging information when the blowing means is activated and deactivated, for example, the amount of movement of the harvested part between the two captured images is such that other factors such as light leaves Based on the difference in the movement amount that is smaller than the movement amount of the object, the harvest target portion is identified in the imaging screen, and the position of the harvest target portion is detected. In addition, in the first characteristic configuration
According to this, the harvesting hand is operated toward the harvesting target part.
In this condition, the air blower provided on the harvesting hand operates.
Also, when the blower is operating, or when the blower is working.
The imaging means provided on the harvesting hand during movement and non-operation
Imaging is activated. Further, according to the first characteristic configuration, the harvest pair
Air blown on the harvesting hand that is operated toward the elephant
Means harvests the working air provided by the air supply means
Blows air toward the target area, and then detects the location of the harvested area.
Based on the output, the harvesting hand is operated to the gripping position of the harvesting target part.
Once created, the supply of working air from the air supply means
Cut the harvesting target to prepare for the harvesting hand.
The grips are replaced and the grips are
To have.

According to the second characteristic construction, in the first characteristic construction, when the air blowing means is operated, the harvesting target portion does not move, but other objects such as leaves move. Then, based on the imaged information when the blowing means is activated in a state where only the other object such as the leaf moves with respect to the harvest target part that does not move in the imaging screen and the two are separated, for example, the shape and color of the harvest target part The position of the harvest target part is detected by identifying the harvest target part from the characteristics, or based on both imaging information when the blowing means is operating and when the blowing device is not operating, for example, between the two captured images, the position of the harvest target part is The position of the harvesting target is detected by identifying the harvesting target within the imaging screen based on the difference in the amount of movement that only other objects such as lighter leaves are moving, while being fixed. .

[0011]

[0012]

[0013]

Therefore, according to the first characteristic configuration of the present invention, other objects such as leaves having a lighter weight than the harvest target portion (fruit portion) are separated from the harvest target portion within the image pickup screen of the image pickup means. , Because the position of the harvest target part is detected based on the image information in the state where the harvest target part is separated from the other objects, conventionally, a part or all of the harvest target part is hidden by other objects such as leaves and stems. In this case, it is difficult to recognize the fruit part based on the shape feature and detect its position because the captured image of the whole harvested part cannot be obtained, or based on the color difference. The problem that the position detection error when recognizing the fruit part and detecting the position becomes large is solved, and thus the harvest target part detection device for fruit and vegetables that can reliably and accurately detect the position of the harvest target part is provided. I got it.
Further, according to the first characteristic configuration, the harvest target portion is approached
Blow from the harvested hand position toward the harvest target area
Image of the harvested part that is blown
Means and blowers are not harvesting hands, for example
Compared to those installed on the body side of industrial vehicles, they are subject to harvesting
Efficient air blowing with a small air flow only to the harvested area
In addition, imaging that captures the harvesting target part to be harvested
Low magnification without the need for expensive zoom mechanism
Inexpensive imaging means can be used. Furthermore, in the first characteristic configuration
According to this, the grip for gripping the harvested part that is not used at the same time
One means for supplying operating air to the air blower and the air blower
Since it also serves as the air supply means of,
Providing operating air supply means separately for the blowing means
In contrast to the need for extra air supply means,
The device can be simplified.

Further, according to the second characteristic configuration, only other things such as leaves move, so that, for example, the other object and the harvested part are moved together, and the amount of movement (the amount of other things such as leaves) is increased. The movement amount is larger than the harvesting target portion), and in the case of detecting the position of the harvesting target portion on the basis of it, it may not be possible to easily determine the magnitude of the movement amount. By the relatively simple process of taking the logical product of both images before and after moving, the image part corresponding to other things such as leaves disappears, the image part of the harvesting part that does not move remains as it is,
It is possible to easily detect the position of the harvesting target portion based on the remaining image, and thus it is possible to obtain a suitable means for implementing the first characteristic configuration.

[0015]

[0016]

[0017]

[Embodiments] The present invention is applied to a harvesting target portion detecting device for detecting the position of a harvesting target portion in an automatic traveling harvesting vehicle for harvesting cucumbers, tomatoes and the like as fruits and vegetables growing in a field. An embodiment applied to the above will be described with reference to the drawings.

As shown in FIG. 1, the harvesting vehicle B is installed in a ridge of a field by driving steering and traveling wheels 2 each including two wheels provided in front and rear positions of a vehicle body 4, respectively. The guide line L based on the detection information of the guide sensor 1 on the vehicle body side which detects the guide line L of the electromagnetic induction type or the like.
Will be guided along. Since the cucumbers A and the like are normally attracted upward along the rods 20 arranged at predetermined intervals, the harvesting work vehicle B positions and stops with respect to each rod 20 to perform harvesting work. Become. Supports 5 on the car body 4
And a rack gear 5a is formed in the column 5 along the up-down direction, and the rack gear 5a is engaged with the arm base 3 that supports the base end portion of the telescopic arm 6 that can be extended and contracted in the lateral direction. The pinion gear 3a is provided in a state of being rotationally driven by a vertical movement motor 3A (see FIG. 3), and by rotating the pinion gear 3a in forward and reverse directions, the arm base 3, that is, the telescopic arm 6 is moved up and down. Is configured.

The tip portion 6A of the telescopic arm 6 functions as a harvesting hand which is operated toward the fruit portion A1 which is the target portion for harvesting the cucumber A or the like, and the harvesting hand, that is, the tip portion of the telescopic arm 6. 6A, as also shown in FIG.
A color CCD camera S1 as an image pickup means for picking up an image of a predetermined area including a fruit part A1 such as a cucumber, and other objects such as leaves A2 and vines that are lighter in weight than the fruit part A1 are within the imaging range of the CCD camera S1 And a blower nozzle 7 as a blower unit for moving the fruit portion A1 existing in the above. Then, the air-blowing nozzle 7 is configured to blow air toward a predetermined range including the fruit portion A1 in a state where the amount of air blow is such that the leaf A2 and other objects are moved but the fruit portion A1 is not moved. It should be noted that conditions such as the pressure of the working air supplied from the compressor 10 described later and the nozzle diameter of the blowing nozzle 7 are set so as to realize the above-described blown air amount state.

The tip portion 6A of the telescopic arm 6 has a fruit portion A.
1 is provided with a gripping portion 9 for gripping the object to be harvested by sandwiching 1 from both sides with the sandwiching nails 9A, and in order to detect whether or not the fruit portion A1 exists between both sandwiching nails 9A, one A photoelectric presence / absence sensor S2 including a light emitter on the side of the sandwiching claw 9A and a light receiver on the side of the other sandwiching claw 9A is provided on the inner surface side of both the sandwiching claws 9A. The operating air is supplied to the grip portion 9, and the sandwiching claw 9A performs the sandwiching operation. A compressor 10 is provided as an air supply means for supplying operating air to the blowing nozzle 7 and the grip portion 9,
The compressor 10 is configured to be switchable between supply of operating air to the grip portion 9 and supply of operating air to the blowing nozzle 7 (see FIG. 3).

Next, the control configuration of the harvesting work vehicle B will be described. As shown in FIG. 3, a controller H using a microcomputer is provided, and the controller H is
The detection information of the guide sensor 1 and the presence / absence sensor S2, and the processing information from the image processing device 12 that has processed the imaging information from the CCD camera S1 are input. On the other hand, from the controller H, the vertical movement motor 3
Drive signals for A, a motor 6A for expanding and contracting for expanding and contracting the expandable arm 6, and a steering motor 2A and a traveling motor 2B for driving the steering and traveling wheels 2 are output. There is. Further, the controller H outputs an on / off signal for turning on / off the operation of the compressor 10 and switches the operating air from the compressor 10 between the grip portion 9 side and the blowing nozzle 7 side. A switching signal is output to the solenoid valve 11 for.

The controller H and the image processing device 12
Utilizing the above, the harvest target position detecting means 100 for detecting the position of the fruit portion A1 based on the image pickup information of the CCD camera S1 is configured. Then, the harvest target position detecting means 100 is based on the image pickup information when the blower nozzle 7 is operating, or based on the image pickup information when the blower nozzle 7 is operating and the image pickup information when the blower nozzle 7 is not operating, The position of the fruit portion A1 is detected.

Next, the position detection of the fruit portion A1 by the harvesting target position detecting means 100 will be described with reference to FIGS. First, when the blower nozzle 7 is not operated, as shown in FIG. 4A, when a part (substantially upper half) of the fruit portion A1 is hidden by the leaf A2 located on the front side, a captured image thereof Is obtained as an image in which the fruit portion A1 and the leaf A2 overlap, as shown in FIG. In this state, for example, in the case of cucumber, fruit part A1 and leaf A2
Since both are greenish, it is impossible to detect the position of the fruit part A1 separately from the leaf A2, and in the case of tomato,
Since the fruit portion A1 is reddish and the leaf A2 is greenish, it is possible to separate the two by color. However, when most of the fruit portion A1 is hidden by the leaf A2, the fruit portion A1
It is virtually impossible to detect the position of. When the blowing nozzle 7 is operated in the above state, the fruit portion A1 does not move and only the leaf A2 on the front side is blown upward as shown in FIG. Fruit part A1 shown in
An image in which the leaf A2 is separated is obtained.

Here, taking a cucumber as an example, the fruit portion A1 of the fruit portion A1 is based on the image pickup information when the blower nozzle 7 is operating (FIG. 5B) and the image pickup information when it is not operating (FIG. 5A). The case of detecting the position will be described. In the two captured images, RGB three primary color information is processed by a predetermined arithmetic expression to extract a region (green region) corresponding to the fruit portion A1 and the leaf A2. That is, the binarization processing is performed such that the pixel value corresponding to the fruit portion A1 and the leaf A2 is 1 and the pixel values of the other portions are 0. Next, when the logical product of the two binarized images is taken, the fruit part A1 part is not moving, so it remains in the image after the logical product processing, but the leaf A2 part is moving, so the logical product is It disappears from the processed image (Fig. 5
(C)). Therefore, the barycentric position of the area of the fruit portion A1 in the image after the logical product processing is obtained and detected as the position of the fruit portion A1.

The procedure of the actual harvesting work by the harvesting work vehicle B having the above construction will be outlined. First, the work vehicle B positions and stops with respect to the rod 20, and the harvesting hand 6A is moved to the fruit portion A1 based on the information from the CCD camera S1.
In the state of being brought close to the fruit and vegetables A containing, the air blow from the air blow nozzle 7 is turned on and off while the telescopic arm 6 is moved up and down, and the picked-up image is obtained at each time of the air blow on and off. Then, based on the position detection information of the fruit portion A1 by the harvesting target portion detecting means 100, the harvesting hand 6A detects the fruit portion A1.
1 is operated to approach, and the presence / absence sensor S2 displays the fruit portion A.
When 1 is detected, operating air is supplied to the gripping portion 9 and the fruit portion A1 is sandwiched by the sandwiching claw 9A. Then, the fruit pattern portion on the upper side of the grasped fruit portion A1 is cut by a cutter (not shown), and the fruit portion A1 is taken into the work vehicle side.

[Other Example] In the above example, a harvester for harvesting vegetables such as cucumber and tomato as fruit and vegetables has been described as an example, but vegetables other than cucumber and tomato, and
It can be applied when harvesting various fruits such as oranges and apples.

In the above embodiment, the image pickup means is constituted by the color CCD camera S1, but other than this, for example,
Various image pickup means such as a black and white CCD camera, a color type and a black and white type electron tube type television camera can be used.

Another embodiment of the position detection of the fruit portion A1 by the harvesting target position detecting means 100, that is, the position of the fruit portion A1 is determined based on the image pickup information (FIG. 5B) when the blower nozzle 7 is operated. The case of detection will be described.
Similar to the above embodiment, the pixel value of the area (green area) corresponding to the fruit portion A1 and the leaf A2 is 1, and the pixel values of the other portions are 0.
To obtain a binarized image. Then, in this binarized image, the region of the fruit portion A1 is discriminated from the regions of the leaves A2 and the like based on the previously stored shape features of the fruit portion A1 (e.g., rounded elongated shape). The center of gravity of the specified area of the fruit part A1 is determined to determine the fruit part A1.
The position is detected as 1.

In the above embodiment, in order to detect the position of the harvesting target portion A1 based on the imaging information when the blowing means 7 is operating and the imaging information when the blowing means 7 is not operating, an area (green area) such as the harvesting target portion A1 is detected. ) Is subjected to an AND process to detect the position of the harvesting target portion A1, and the position of the harvesting target portion A1 is detected based on the imaging information when the blower 7 is operated. The position of the harvest target portion A1 is identified based on the shape feature of the harvest target portion A1 and the position of the harvest target portion A1 is detected. However, the present invention is not limited to such a process, and various image processing means may be used. Can be used.

In the above embodiment, the air blowing means is constituted by the air blowing nozzle 7 for blowing the air supplied from the air supplying means (compressor 10), but the present invention is not limited to this, and the air blowing means is rotationally driven by, for example, a motor or the like. It may be a fan type blower.

Further, in the above embodiment, the amount of air blown by the air blower (the nozzle 7 for blowing air) is set so that other objects such as the leaves A2 are moved but the target portion (fruit portion A1) is not moved. For example, when other things such as leaves A2 are relatively heavy fruits and vegetables, the amount of air blown is increased to a state where the target part (fruit part A1) moves to ensure that the other parts are harvested from the target part A1. You may make it isolate | separate into. In this case, when the position variation of the harvest target portion (fruit portion A1) is large, and when the position of the harvest target portion A1 is obtained by performing a logical AND process on the images before and after the operation of the blower unit (blast nozzle 7) described above. When the detection error of 1 is not negligible, for example, the position of the harvesting target portion A1 is obtained based on the shape feature of the harvesting target portion A1 from the image when the blowing means (blowing nozzle 7) is operating.

In the above embodiment, the image pickup means S1 and the air blowing means 7 are provided in the harvesting hand 6A. However, the invention is not limited to this, and the image pickup means S1 and the blowing means 7 are provided separately from the harvesting hand 6A, for example, with a zoom mechanism. The means S1 may be installed on the base end side of the telescopic arm 6, or a fan-type air blowing means set in the vehicle body 4 of the work vehicle B in an obliquely upward air blowing direction.

In the above embodiment, the gripping portion 9 for gripping the harvesting target portion is gripped by sandwiching the harvesting target portion A1 from both sides with the sandwiching claws 9A, but the invention is not limited to this. A bendable arm may be wound around and gripped around the entire outer peripheral surface of the portion A1.

In the above embodiment, the air supply means (compressor 10) is configured to be switchable between the supply of operating air to the grip portion 9 and the supply of operating air to the blower means 7. However, the present invention is not limited to this. Instead, separate air supply means may be provided.

Further, in the above embodiment, the apparatus for detecting a target portion for harvesting fruits and vegetables according to the present invention is applied to an automatic traveling type harvesting work vehicle B which travels through a furrow in a field where cucumbers and the like as fruits and vegetables are grown. However, the harvesting vehicle is not limited to an automatic traveling harvesting vehicle, and may be a manually operated harvesting vehicle, or an automatic harvester provided on a moving body that moves in the air above the field. Various embodiments such as application are possible.

It should be noted that reference numerals are given in the claims for convenience of comparison with the drawings, but the present invention is not limited to the structures of the accompanying drawings by the entry.

[Brief description of drawings]

[Figure 1] Front view of an automatic traveling harvesting vehicle

FIG. 2 is a perspective view showing a harvesting hand.

FIG. 3 is a block diagram of a control configuration.

FIG. 4 is an external view of a part to be harvested before and after the operation of the air blower.

FIG. 5 is an explanatory diagram showing image processing for detecting a harvest target part.

[Explanation of symbols]

A1 Harvest Target S1 image pickup means 100 Harvest target position detection means A2 Others 7 Blower means 6A Harvesting hand 9 grip 10 Air supply means

Continuation of the front page (56) References JP-A-6-276827 (JP, A) JP-A-4-184573 (JP, A) JP-A-6-141649 (JP, A) Japan Agricultural Meteorological Society 1990 National Convention・ Proceedings of the 28th Annual Meeting of the Japan Society for Biological Environmental Regulation, Japan Agricultural and Meteorological Society, Japan Bioregulatory Society, 1990, 148-149, Robotic harvest of cucumber fruit (58) Fields investigated (Int .Cl. ⁷ , DB name) A01D 46/00-46/28

Claims (2)

(57) [Claims] 1. An image pickup means (S1) for picking up an image of a predetermined range including a harvested portion (A1) of fruit and vegetables, and the image pickup means (S).
1) A harvesting target portion detection device for fruit and vegetables provided with harvesting target position detecting means (100) for detecting the position of the harvesting target portion (A1) based on the imaging information of 1), wherein the harvesting target portion ( Others that are lighter in weight than A1) (A2)
Is provided with a blower unit (7) for moving so as to move away from the harvesting target portion (A1) existing in the imaging range of the imaging unit (S1), and the harvesting target position detection unit (100) is The harvest target part (A) based on the imaged information when the air blowing unit (7) is operating, or based on the imaged information when the air blowing unit (7) is operating and the imaged information when the air blowing unit (7) is not operating.
Configured to detect the position of 1), wherein the imaging means (S1) and said blowing means (7) comprises
Harvesting han that is operated toward the harvesting target part (A1)
(6A) for holding the harvesting target portion provided in the harvesting hand (6A)
For operating the gripping part (9) and the blowing means (7)
The air supply means (10) for supplying air is the grip portion.
Supply of operating air to (9) and the blowing means
The supply of operating air to (7) can be switched freely.
A device for detecting the target part of the harvest of fruits and vegetables that has been made. 2. The blower means (7) is the other object (A2).
The fruit and vegetable harvesting according to claim 1, which is configured to blow air toward a predetermined range including the harvesting target portion (A1) in a state in which the amount of air is moved but the harvesting target portion (A1) is not moved. Target part detection device.