JP3443865B2 - Fruit harvester - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention, the fruits and vegetables of the harvest target
The present invention relates to a fruit vegetable harvesting machine that identifies and harvests by performing image processing . [0002] Conventionally, fruit and vegetables are cucumbers. In the detection of cucumbers, the shape of the cucumbers is imaged by a camera while self-propelled along the cultivated plantations, and the image is obtained by this imaging. The detected image signal is binarized to generate a binarized image, the ratio between the height and width of the outline of the binarized image is calculated, and whether the image is a cucumber or a leaf is calculated based on the calculated ratio. Is set to detect cucumbers
I was [0003] In a fruit vegetable harvesting machine, for example, fruits and vegetables are cucumbers. When the cucumbers are detected and harvested, the cucumbers are self-propelled along the cultivated plantations. Is captured by a camera, a detected image signal obtained by the imaging is binarized to generate a binarized image, and the ratio between the height and width of the outline of the binarized image is calculated. It is determined whether the cucumber is a cucumber or a leaf other than the cucumber based on the calculated ratio, and only the cucumber is harvested. [0004] In this cucumber harvesting operation, when the cucumber is picked up with a leaf when the cucumber is picked up with a camera, the binarized image is an image in which the cucumber and the leaf are integrated. Because of this, the vertical / horizontal ratio changed, and cucumber was sometimes incorrectly detected as not cucumber.
This is intended to be solved. SUMMARY OF THE INVENTION [0005] The present invention provides a method for setting
Harvest fruits and vegetables that have a continuous width in the set range in the vertical direction
Vegetable harvester that captures a predetermined area including the fruits and vegetables.
A camera 3 for imaging, and an image captured by the camera 3
Binarization processing means for binarization, and the binarization processing means
Pixels in vertical and horizontal pixels forming the obtained binarized image
Counting means for counting the number of pixels in the horizontal direction for the group and vertical
Counting means for counting the number of pixels in a direction, and the counting means
The pixels counted within the setting range in the horizontal direction are
When counted within the setting range, the pixel group is
Judging means for judging a vegetable;
The fruits and vegetables corresponding to the group of images judged to be elephant fruits and vegetables are collected.
And a harvesting unit for harvesting.
The structure of the harvester . When a cucumber is detected and harvested by a fruit and vegetable harvester, a predetermined area is imaged by the camera 3 while traveling along the cultivated plantations, and the image is detected by the image pickup. binarizing the image signal, the presence or predetermined number of lines transverse width in a portion of the binarized pixel group is within a predetermined range continuously in the longitudinal direction, the pixel group of crop object Cucumber is determined. In cases other than the above, it is determined that the stem or leaf is other than cucumber. With these determinations, only cucumber is harvested. [0007] The present invention relates to a fruit vegetable constituted as described above.
Because it is a harvester, fruits and vegetables overlap with leaves and fruits
Even if the image is captured without showing the aspect ratio of the whole class,
Fruits and vegetables to be harvested can be determined and harvested. An embodiment of the present invention will be described below with reference to the drawings. The example of the figure is a diagram showing a fruit and vegetable harvesting machine 1. For example, a shape of a fruit and vegetable cucumber 2 is imaged by a camera 3, and the imaged image is binarized. A detection method for detecting whether the number is 2 will be described. In the fruit and vegetable harvesting machine 1, the shape of the cucumber 2 for harvesting the cucumber 2 and the shape of the cucumber 2 to be harvested, for example, at wavelengths of 850 nm and
A visual mechanism 7 composed of multi-plate CCD cameras 3 and 3 for imaging based on light of nm and a position sensor 6 for detecting the position of the cucumber 2 is provided. In a cultivation field using the fruit and vegetable harvester 1, the cucumber 2 tree 9 is grown on a support 8 provided diagonally. The manipulator 5 includes a facing tree 9
Frame 10 inclined so as to be substantially parallel to the inclination of
A base 12 mounted on the inclined frame 10 so as to be able to move up and down along a guide rail 11, a main body 13 provided on the base 12 so as to be rotatable horizontally, and the manipulator 5. An arm 14 forming an articulated shape provided, and a hand device 15 for holding and cutting the cucumber 2 provided at the tip of the arm 14
It is a configuration consisting of The inclined frame 10 is mounted on the traveling section 4 using a hinge 16, and the back side is supported by a support link 17. The lower end of the support link 17 is provided with a long hole 1 of a support plate 18.
9 is mounted at an arbitrary position, and the mounting position is adjusted so that the inclination angle of the inclined frame 10 can be arbitrarily adjusted. By operating each part of the manipulator 5, the cucumber 2 to be harvested with the hand device 15 is sandwiched, and the fruit of the cucumber 2 is cut. The visual mechanism 7 is, as shown in FIGS. 1 and 3,
A camera lens 20 is provided at the front, and a spectroscope 21 'is provided at the rear side of the camera lens 20.
A half mirror 21 is provided at the front of 1 ', and each reflector (A) 22a,
(B) 22b and (C) 22c are provided, and each of the reflectors (A)
An interference filter (A) 23a for a wavelength of 850 nm is provided on the rear side of 22a and (B) 22b, and an interference filter (B) 23b for a wavelength of 550 nm is provided on the rear side of the reflector (C) 22c. A) 23a and (B) 23b are provided with multiple CCD cameras 3 and 3 at the rear side, respectively, and are configured to image a predetermined range with light at wavelengths of 850 nm and 550 nm. The visual mechanism 7 scans the field of view of each of the multi-plate CCD cameras 3 and 3 horizontally and vertically to measure the distance to the cucumber 2 of the object, and the respective multi-plate CCD cameras. The illuminance sensor 24 detects the illuminance when the images captured by the cameras 3 and 3 are captured. Each of the multi-plate CCDS cameras 3 and 3 and the position sensor 6
And various detections detected by the illuminance sensor 24 are performed by the control device 2
5 to the CPU 27 via the input circuit 26,
The PU 27 analyzes and processes various commands through the output circuit 28 to operate the manipulator 5 and the like. The control device 25 is provided in the visual mechanism 7. Next, the control of the cucumber detection method from the binarized image in the embodiment of the present invention configured as described above is shown in FIG.
This will be described with reference to the flowchart of FIG. The cucumber 2 harvesting operation is started (step 101), the imaged and binarized binarized images are labeled, and the number of pixels of each labeled binarized image is calculated (step 10).
2) The number of pixels (Gx) of each label is (Gx) ≧ 300
Is detected (step 103), and if NO is detected, it is deleted (step 104) and the process returns to step 102. If YES is detected, the Feret length ratio of the image (F
A) is calculated (step 105), and the Feret length ratio (FA) is the ratio between the vertical (vertical) length (fv) and the horizontal (horizontal) length (fh) of the image as shown in FIG. Is f
Calculated as v / fh. Feret length ratio (FA) calculated
Is detected whether (FA) ≧ 2.2 (step 106),
When YES is detected, it is determined that the cucumber is the cucumber 2 (step 107). If NO is detected, the horizontal intercept length (LA) is calculated (step 108), and the number of pixels (Li) of the horizontal line is 5 ≦ (Li) ≦ 20.
Is detected (step 109), 5 and 20 are the number of pixels in the horizontal line of the image. If YES is detected |
It is detected whether (Li + 1) − (Li) | ≦ 2, and it is detected whether the difference between the number of pixels above and below the horizontal line is 2 or less (step 110). Count number (Gx) = Gx
+1 (step 111), the count number (Gx) is
(Gx) ≧ 30 is detected, and it is detected whether the difference between the number of upper and lower pixels detected in step 110 is 2 or less than 2 or more than 30 pixels continue (step 112). If YES is detected, FLAG = 1 is a configuration that is determined to be cucumber 2. (Step 107). If NO is detected, FLAG = 0 and it is determined that the stem and leaf are other than cucumber 2. The FLAG is detected as shown in FIG. 6 and is the (i) -th pixel number (Li) of a predetermined horizontal line.
If a line (the number of pixels in the vertical direction) having a horizontal width (number of pixels) of 5 to 20 dots is present continuously for 30 or more lines, FLAG = 1 is detected. Is detected. Predetermined pixel stage (i) = E
ND is detected (step 113), and is deleted when YES is detected (step 114). If NO is detected, (i) = (i) +1 is set (step 11).
5), the structure proceeds to step 108. NO at steps 109 and 110
Is detected, the count number (Gx) is set to 0 (step 116), and the process proceeds to step 113. FIGS. 7A to 9B are images in which the cucumber 2 is recognized from the images on which noise clearing and labeling have been performed, and FIGS. Is determined. Next, control of noise component elimination will be described with reference to the flowchart of FIG. Starting (step 201), the central pixel (f) in the horizontal direction of the binarized image
0) The position is searched (step 202), the periphery of the central pixel is divided into eight, and these eight divided sections are
A search is made with a score attached, such as 1 (step 20).
3) The number 1 is counted (step 204), and it is detected whether count ≧ 4 (step 205). If YES is detected, the central pixel (f0) = 1 (step 20).
6) It is detected whether all the pixels have been searched (step 20).
7), the process is terminated when YES is detected (step 208). If NO is detected in step 205, the central pixel (f0) is set to 0 (step 209), or the process proceeds to step 207. When NO is detected in step 207, the process returns to step 202. Next, the labeling control is performed according to the flow chart of FIG.
This will be described with reference to the chart. Start (Step 3)
01) The horizontal central pixel position of the binarized image is searched (step 302), and the search order of the pixels is performed as shown in FIG. fi indicates a horizontal pixel, fj indicates a vertical pixel, and fij indicates a central pixel. It is detected whether the central pixel (fij) = 1 (step 30).
3) If NO is detected, the process returns to step 302. If YES is detected, it is detected whether all adjacent points are 0 (step 304). If YES, fi is detected.
j is assigned a new label (step 305), and step 3
02. If NO is detected, it is detected whether all adjacent points have the same label (step 306). If NO is detected, fij is set as the first label (step 30).
7) The other and the foremost label are recorded as the same label (step 308), and it is detected whether or not the previous pixel has been searched (step 309).
Returning to step 2, if YES is detected, the concatenated label is rewritten (step 310), and the process ends (step 311). If YES is detected in step 306, fij is given the same label (step 312), and the process proceeds to step 309. Next, the control for extracting the center line of the binarized image will be described with reference to the flowchart of FIG. The process is started (step 401), and the label is searched (step 40).
2), fi indicates horizontal pixels, fj indicates vertical pixels, and fij indicates central pixels. Pixel fi,
j-1 = 0.fi, j = label is detected (step 4
03), if NO is detected, the flow returns to step 402, and Y
If ES is detected, j = j + 1 (step 404),
fi, j-1 = label fi, j = 0 is detected (step 405), and if NO is detected, the process returns to step 404. If YES is detected, the center point (S) of the both-end image is set (step 405). 406), j = end is detected (step 407), and if NO is detected, j = j + 1 (step 408), and the process returns to step 402. If YES is detected, it is detected whether i = end (step 409),
If NO is detected, i = i + 1 (step 410),
The configuration returns to step 402, and ends when YES is detected (step 411). Next, the calculation of the direction of the center line of the binarized image will be described with reference to the flowchart of FIG. Star
(Step 501), a start point of the center point (S) is searched (step 502), a search is made in a counterclockwise direction (step 503), and it is detected whether or not the center pixel (fi) = S (step 504). If YES is detected, the count number (Cg) is equal to Cg + 1 (step 505), the tracked mark is set to the center pixel (f0) (step 506), and the horizontal pixel (fi) is set to the center pixel (f0). ) (Step 507), and returns to step 503. If NO is detected in step 504, fi = f8,
Or fi = tracked mark is detected (step 50)
8) If NO is detected, the process returns to step 503. If YES is detected, Cg ≧ 5 is detected (step 509), and if YES is detected, the coordinates of the upper and lower ends are recorded (step 510). ), Bond center line length (T) = |
xe-xt | / ye-yt (step 511), or the length of the bonding center line (T) ≦ 0.42 (tan22.5 ゜ = 0.
42) is detected (step 512), and if YES is detected, it is detected whether all center lines have been searched (step 51).
3), the configuration ends when YES is detected (step 514). When NO is detected, the process returns to step 502. N in steps 509 and 512
When O is detected, the center line is deleted (step 51).
5) Return to step 502. Next, the connection control of the center lines will be described with reference to the flowchart of FIG. The process is started (step 601), and the lower end of the center line is searched (step 602).
The upper end of the lower center line is searched (step 603), and it is detected whether there is a lower center line (step 604). If NO is detected, the number of pixels of the center line is calculated (step 60).
5) The number of pixels (the number of dots) ≧ the reference pixel number hi forming the center line that is changed according to the position of the image is detected (step 606). If YES is detected, the top pixel is marked with cucumber 2 marks. (Step 607), the distance from the y-axis center line is calculated (Step 608), and it is detected whether or not all the center lines have been searched (Step 609). If YES is detected, the process is terminated (Step 610). . When NO is detected, the process returns to step 602. Step 6
If NO is detected in step 06, the process proceeds to step 609. Here, even if the cucumbers 2 have the same shape and size, the size (center line length) on the binarized image differs depending on the degree of separation from the optical axis center of the multi-plate CCD camera 3. Therefore, it is necessary to appropriately change the reference pixel number hi. If "YES" is detected in the step 604, a center line in the same label is detected (step 611).
If O is detected, it is detected whether there is an image between the upper and lower center lines (step 612), and if NO is detected, step 60 is performed.
3, when YES is detected, the directionality of the connection line is calculated (step 613), and it is detected whether or not the coupling center line length (T) ≦ predetermined value (d) (step 614),
If YES is detected, a linkable mark is made (step 6).
15) It is detected whether or not all the lower center lines have been searched (step 616), and if YES is detected, it is connected to the connecting line of Tmin (step 617), and the process returns to step 603. When NO is detected in step 614, the process proceeds to step 616. If NO is detected in step 616, the lower center line is connected (step 61).
8), the structure returns to step 613. The operation of the above embodiment will be described below. In the vegetable harvesting machine 1, for example, cucumber 2 is a cucumber, and when the cucumber 2 is detected and harvested, while the cucumber 2 is self-propelled along the cultivated plantations, The shape is imaged by the multi-plate CCD cameras 3 and 3 of the visual mechanism 7 with light of a predetermined wavelength. A detected image signal obtained by this imaging is input to the CPU 27 and is binarized to generate a binarized image. The binarized image is formed by a group of pixels formed by a set of pixels. The Feret length ratio of the maximum length in each of the vertical direction and the horizontal direction is calculated, the calculated Feret length ratio is compared with a predetermined value, and it is detected that the calculated Feret length ratio is equal to or more than the predetermined value, Alternatively, the horizontal width (the number of pixels) of each pixel is within a certain range, and a portion where the vertical lines (the number of pixels) within this certain range continuously show a predetermined value or more is included in a part of each pixel group. Is detected, it is determined that the cucumber 2 is the harvest target. Otherwise, it is determined that the stem or leaf is other than cucumber 2. According to these determinations, only cucumber 2 is harvested.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall front view of a fruit and vegetable harvester in a vegetable and vegetable cultivation field. FIG. 2 is a block diagram. FIG. 3 is an enlarged side sectional view of a visual mechanism. FIG. 4 is a flowchart. FIG. 5 is a diagram illustrating a Feret length ratio calculation. FIG. 6 is a diagram for calculating a horizontal intercept length. FIG. 7A shows an image after noise clearing and labeling. (B) Recognition result diagram. (C) Feature table. FIG. 8A shows an image after noise clearing and labeling. (B) Recognition result diagram. (C) Feature table. FIG. 9A is an image after noise clearing and labeling. (B) Recognition result diagram. (C) Feature table. FIG. 10 is a flowchart. FIG. 11 is a score distribution chart. FIG. 12 is a flowchart. FIG. 13 is a labeling procedure diagram. FIG. 14 is a flowchart. FIG. 15 is a flowchart. FIG. 16 is a flowchart. [Explanation of Signs] 3 Camera (multi-plate CCD)

Claims (1)

(57) [Claims] [Claim 1] The width of the set range is continuously set in the vertical direction
A fruit harvester for harvesting fruits and vegetables having
A camera 3 for capturing an image of a predetermined area including vegetables,
Binarization processing means for binarizing an image captured by
Forming a binarized image obtained by the binarization processing means;
Count the number of pixels in the horizontal direction for the pixels in the vertical and horizontal pixels
Counting means and counting for counting the number of pixels in the vertical direction
Means and a number within the set range in the horizontal direction by the counting means.
When the number of pixels obtained is counted within the set range in the vertical direction.
Determining means for determining a group of pixels as fruits and vegetables to be harvested;
Image group judged to be fruits and vegetables to be harvested by judgment means
Harvesting unit that takes in and harvests fruit and vegetables equivalent to
Fruit harvester characterized by the following.