JP2010207118A - End effector for harvesting fruit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an end effector for harvesting fruit, as a robot, having the function of recognizing targets in a three-dimensional space with many complicated obstacles, and capable of selecting highly ripened fruit alone and harvesting the fruit. <P>SOLUTION: The end effector for harvesting fruit includes: a manipulator 3 having six kinds of degree of freedom, i.e. transverse turning, vertical movement, vertical turning, the end effector (wrist 6)' s transverse shaking, and the wrist turning; a three-dimensional visual sensor 4; and a central processing unit 7. In the effector, an arm 5 is set up stretchably/contractibly on the manipulator 3, and the tip of the arm 5 has: a U-shaped fitment 11 and a chute 12 for taking in fruit for harvesting the fruit; a small-fruit-twig press 14 for pressing the fruit's small-fruit twig (W2); a third driving means M3 for open/close-driving the press 14; a second driving means M2 for twisting the U-shaped fitment 11; and a first driving means M1 for shaking right and left the two driving means and the U-shaped fitment 11, etc. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an end effector for a fruit harvesting robot, and more particularly to a fruit harvesting end effector corresponding to the hand portion of a cherry tomato harvesting robot that automatically harvests cherry tomatoes.

  Production of fruit (fruit vegetables) such as cherry tomatoes is often carried out manually with visual judgment, and mechanization has not been extended. According to the Ministry of Agriculture, Forestry and Fisheries 2006 management statistics by product, in the case of facility-grown large tomatoes, the working time per 10a is 933 hours, of which 298 hours (32.0% of the total working hours) are required for harvesting work. Yes. In the case of cherry tomatoes, the working time per 10a is 1,451 hours, and the working time reaches 3992 hours with a planted area of 27.5a per house. Because cherry tomatoes have a large number of grains, the harvesting work accounts for the largest share of 37.7% of working hours. Bunch harvesting is sometimes performed to harvest the whole bunch, but cherry tomatoes ripen red from the top of the fruit bunch, and when the red cram goes to the bottom, the top is often ripped, and many are harvested one by one. Yes.

  The mechanization of tomato harvesting has been developed for processing tomatoes that are harvested all at once and is popular in the United States and other countries, but the harvesting of raw tomatoes depends on human hands. For this reason, research on tomato harvesting robots has been conducted since around 1982, but it has not been put into practical use. In the case of fruit (fruit vegetables) harvesting, depending on the crop and cultivation mode, the leaves and struts may be obstructed, making it difficult to harvest, or the fruit may be hidden behind the leaves and leaves. For this reason, in recent years, research has been conducted on fruits that are suitable for robotization and fruits in cultivation styles, research that recognizes and avoids obstacles by processing 3D visual sensor images, and hides obstacles. Research has been conducted to scan and harvest fresh fruits from a different perspective. The three-dimensional visual sensor used for harvesting tomatoes and the like is a two-wavelength type that scans near-infrared and red laser beams. In the present invention, this two-wavelength three-dimensional visual sensor is used. This will be described later in the section of the best mode for carrying out.

As an example of a conventional fruit harvesting apparatus, a fruit trap supporting member that is provided with a fruit catching portion provided with a fruit storage space with an opening for harvesting at the upper portion and supports the fruit stem extending from the storage space at a predetermined position, It is provided in the fruit catching part so as to be switchable between a supporting action position protruding to the intake side and a retreat position retracting to the side away from the fruit intake so that the fruit to be harvested is taken into the fruit storage space. A fruit harvesting device provided with fruit trapping guidance means for guiding the fruit trapping part, and a fruit bellows member operating means for operating the bellflower support member to the support action position side in order to support the fruit fruit trap of the incorporated fruit. Known (Patent Document 1). In this document, as shown in FIG. 8, a television camera 27 as an operation means for confirming a certain direction of the harvest target fruit M is attached to the upper end portion of the third arm 7c in a state where the optical axis is parallel to the arm expansion / contraction direction. A monitor 28 that captures the image of the camera 27 is provided, a light pen 29 is provided for selecting the harvest target M among the fruits projected on the monitor 28, and the fruit catcher G is set at a set distance from the harvest target fruit M. Infrared proximity sensor 30 for detecting the proximity of the sensor is provided at the upper end of third arm 7c, and fruit selection information, detection information from proximity sensor 30, arm driving motors m ₁ and m ₂ and turning A control device C is provided that outputs an operation command based on the rotation information of the table 5 and the control information stored in advance.

  In addition, the fruit connected to the tip of the arm is designed to reduce the weight to simplify the configuration for guiding the robot hand and to reduce the weight of the cutter that cuts the flesh in place without damaging the fruit. A pair of arc strips that are provided in the cylindrical case for intake so as to be able to be driven out and retracted and that can be driven to move in and out in a predetermined position in accordance with the projecting operation, and that are moved relative to each other in the longitudinal direction to cut. A fruit harvesting robot hand provided on the case side so as to be able to move back and forth freely so as to project and move to a cutting position with respect to the curd at the predetermined position, A fruit harvesting robot hand is known in which each of the pair of blades is formed in a shape that is curved in a longitudinal view (Patent Document 2).

No. 04-11162 Japanese Utility Model Sho 61-80627

  In the tomato harvesting robot, which has been studied in the past, when the fruit is peeled off, the stick is often removed. Raw tomatoes need to be harvested with straw attached, and it was a challenge to harvest with this straw. The inventor according to the present application has conducted research on a tomato harvesting robot hand using a robot hand as shown in FIGS. 23 (A) and 23 (B). This robot hand has a motor m1 attached to the lower portion of the tip of the manipulator 23, a U-shaped bracket 21 provided at the tip of the manipulator 23, a chute 22 attached to the U-shaped bracket 21, a fruit storage container 25, The small pedicel presser 24 and the like can be rotated (left and right) greatly by moving the hand left and right by the motor m1. Further, the motor m2 can rotate (wrist rotation) the U-shaped bracket 21 and the chute 22, the fruit storage container 25, the small fruit infarct retainer 24 and the like attached to the U-shaped bracket 21 around the axis in the fruit direction. It is. In addition, the motor m3 uses a link mechanism to place the fruit in the U-shaped bracket 21 and take it in with the pedicle infuser 24, hook the peduncle with the U-shaped bracket 21 and tear it off and harvest it with a basket. I can do it. The taken fruit is put into a chute 22 provided in the lower part of the U-shaped metal fitting 21, and is put into a fruit storage container 25 to be harvested.

  However, since the structure of the robot hand shown in FIG. 23 is complicated and slightly large, it is difficult to use in a narrow space between obstacles such as foliage, and can be used in a specific suspended mobile cultivation facility. If used in this house, the end effector may damage the stem. Further, as an end effector of a manipulator, it is necessary to handle a delicate crop with a complicated shape, that is, to handle a fruit between three-dimensional stems and leaves, or to handle a soft seedling without damaging it. In order to have a function to recognize an object, that is, to substitute a robot for a detailed work while a person sees and judges the object, the shape, position, cram degree, etc. of the object are recognized. When the ability to determine behavior is necessary and recognition is performed by image processing, it must be able to cope with greenhouses and outdoors where lighting conditions change. In general, in order to be a mobile robot, that is, to move between crops and perform unmanned work, a moving mechanism such as a moving unit having rough terrain traveling performance, safety, and an automatic traveling function is required. It is also necessary to be able to carry out farming work according to the season.

  The present invention has been made in order to address the above-described problems, and has a function of recognizing a target in a three-dimensional space with many complicated obstacles. An object is to provide an end effector for fruit harvesting of a robot that can be used.

  The invention described in claim 1 includes a traveling unit, a manipulator (3) having six degrees of freedom of left / right turning, up / down movement, up / down turning, left / right swing of the end effector (wrist) (6), and wrist rotation, Harvesting a robot provided with a three-dimensional visual sensor (4) and a central processing unit (7), and provided with arms (5) on the manipulator (3) so as to be extendable and retractable so that fruits are taken in at the tip. In the end effector for use, the U-shaped bracket (11) and the chute (12) to be harvested for harvesting fruit at the tip of the arm (5) of the manipulator (3) and the U-shaped bracket (11) And a third drive for opening and closing the mouth of the U-shaped bracket (11) with a link mechanism. Means (M3); Twist the letter-shaped bracket (11) and chute (12) (left and right rotation of the wrist), the second drive means (M2), the two drive means (M3, M2), the U-shaped bracket (11) and the chute ( And 12) a first driving means (M1) that swings left and right (wrist left and right swing).

  According to the second aspect of the present invention, the U-shaped metal fitting (11) and the small fruit infarct presser (14) are provided between the first drive means (M1) and the second drive means (M2). And a second drive means (M2) is provided with a power transmission mechanism that swings left and right.

  According to the third aspect of the present invention, the harvesting path of the end effector includes the number of three-dimensional pixels such as the main stem, leaves, immature fruit, fruit belly, small fruit belly, etc. of the received light image of the three-dimensional visual sensor (4); It is characterized in that it is determined by counting the number of pixels of the edible bellows at the position corresponding to the letter-shaped bracket and other red immature fruits on the path.

  If the end effector for fruit harvesting is used as the above-mentioned means, only fruits with high cram degree located in a complex three-dimensional space will be harvested, and efficiently avoiding the foliage, struts or immature fruits in the complex three-dimensional space. Can be harvested. For this reason, the work time for harvesting cherry tomatoes, which has conventionally been particularly troublesome, can be greatly reduced. Moreover, if the harvesting operation time is shortened, it is possible to increase the planting area.

It is a whole side view of the robot incorporating the end effector for fruit harvesting of this invention. It is a side view of the robot used for the end ejector of the present invention. It is a rear view of the robot used for the end ejector of the present invention. It is the front view seen from the front of the robot in which the end ejector of the present invention is used. It is a figure which shows the structure of the optical system of the three-dimensional sensor of the end ejector of this invention. The flowchart which irradiates a target flower axis with a laser beam and images reflected light and processes a personal computer image in the optical system of the end ejector of the present invention is shown. It is a flowchart which shows the signal flow of the 2 wavelength type three-dimensional visual sensor used with the end ejector of this invention. It is a graph which shows the spectral reflection characteristic of the cherry tomato measured by the 2 wavelength three-dimensional visual sensor used with the end projector of this invention. It is a near-infrared image and is a figure which shows the intensity | strength of near-infrared laser reflected light. It is a red image and is a figure which shows the intensity | strength of red laser reflected light. It is a figure which shows the distance image to the position of the fruit measured with the galvanometer. It is the figure which divided the area | region by comparing a red image and a near-infrared image for every pixel. Furthermore, it is the figure which performed image processing and divided | segmented into the center of each fruit, a fruit stem, a foliage, etc. It is a side view which shows the structure of the end effector for fruit harvesting of this invention. It is a figure of the end effector for fruit harvesting of the state which cut the pedicle of the fruit taken in in the U-shaped metal fitting with the pedicle infusing. FIG. 16 (A) is a plan view showing the configuration of the fruit harvesting end effector of the present invention, and FIG. 16 (B) is a partially enlarged plan view. FIG. 17A is a view showing a left inclined posture, FIG. 17B is a view showing a posture in which the end effector is straightened and harvested, and FIG. FIG. 17D is a diagram illustrating a posture in which the end effector is swung to the left by 110 ° and tilted by 45 ° for harvesting. It is a figure which shows the relationship of each inclination of said FIG. 17 (A)-(D) in the case of harvesting a fruit. It is a flowchart until it selects a path | route in case an end effector goes to harvest a fruit by the image processing of a three-dimensional visual sensor. It is a figure which shows the case where the fruit berry ingested in the U-shaped metal fitting 11 is pressed by the pedicle infusing with a U-shaped metal fitting, pulled and separated by delamination for harvesting. It is a figure which shows the case where a U-shaped metal fitting is pressed on a pedicle infuser, a bend is applied to the pedicle, and it is made to detach | leave by delamination. The pedicle is pressed in a state of being bent between the U-shaped bracket and the pedicle infuser, and in this state, the bending is applied and the layers are separated by delamination. It is a figure which shows the structure of the robot hand tip part for harvesting with the conventional basket.

FIG. 1 is an overall side view of a robot 1 incorporating a fruit harvesting end effector according to the present invention. The robot 1 can travel on the rail R. The tomatoes W were planted at 45 cm intervals between the plants, and the tomatoes W were attracted with hemp string with a 45 ° inclination. 2 is a side view of the robot 1, FIG. 3 is a rear view of the robot 1, and FIG. 4 is a front view of the robot 1 as viewed from the front.
The robot 1 includes a pedestal 2 to which wheels X and X and auxiliary wheels Y and Y traveling on a rail R are attached, a manipulator 3 corresponding to a shoulder and a waist, a three-dimensional visual sensor 4 corresponding to an eye, and a telescopic arm 5. A first half 1A having an end effector 6 corresponding to the hand, and a second half 1B in which a computer (central processing unit: CPU) 7 and a television monitor 8 are provided with a power supply drum 9 and a drive mechanism 10. ing. The end effector 6 is attached to the tip of the arm 5 of the manipulator 3.

  The manipulator 3 has six degrees of freedom in addition to the expansion and contraction of the arm 5, left / right turning, up / down movement, up / down turning, left / right swing of the end effector (wrist), and wrist rotation. Among them, a 100 W AC servo motor was used for the left / right turn, up / down movement, up / down turn, and arm extension / contraction of the manipulator 3, and a 4.5 W DC servo motor was used for the left / right swing of the end effector (wrist). Further, a servo motor for prop is used to rotate the wrist. The approach and retreat of the end effector 6 of the present invention to the fruit is due to the expansion and contraction of the arm 5 of the manipulator 3.

The three-dimensional visual sensor 4 is attached to the manipulator 3 as shown in FIGS. 3 and 4, and the position of the three-dimensional visual sensor 4 is changed by using the left / right turning, up / down movement, and up / down turning of the manipulator 3 at two kinds of positions. It can be scanned. The three-dimensional visual sensor 4 is a two-wavelength three-dimensional visual sensor.
(1) It is possible to measure the three-dimensional shape of a crop that is difficult to recognize with a two-dimensional image of a video camera.
(2) A near-infrared image (two-dimensional) and a red image (two-dimensional) can be obtained in addition to the distance image (three-dimensional).
(3) Since it is an active range finder system, it is not easily affected by the brightness of ambient light and the color temperature.
Fruit harvesting by the robot's end effector 6 requires not only the three-dimensional position of the fruit but also the surrounding obstacles to be harvested so that they do not collide. It has been broken. In addition, if each part of the crop can be recognized in this way, it can be applied to operations such as defoliation and attraction. Therefore, in the present invention, the red and near-infrared laser beams are superposed into a single beam, and this is scanned in the vertical and horizontal directions to obtain three-dimensional information. This uses a two-wavelength three-dimensional visual sensor 4. . As will be described later, the path of the end effector 6 is determined by the number of pixels of an image formed on the two-wavelength three-dimensional visual sensor 4.

  The two-wavelength three-dimensional visual sensor 4 forms an image of reflected light from an object on a light receiving surface of a PSD (Position Sensitive Device) with a lens, and measures a distance by a triangulation method. The two-wavelength three-dimensional visual sensor 4 shown in FIGS. 3 and 4 is a stray light elimination type PSD, and is composed of a light projecting mirror, a light receiving mirror, a red filter, a lens, a cold filter, an infrared laser, and a galvanometer. In the PSD, the ratio of the current output from the two electrodes (anode A and anode B) varies depending on the light receiving position. The distance can be measured by the triangulation method using the characteristics. A red laser having a wavelength of 685 nm and a near infrared laser having a wavelength of 830 nm were used. The red filter is mounted on the lens in order to block components of sunlight that have a shorter wavelength than red. That is, as shown in FIG. 5, the cold filter reflects the near-infrared laser and transmits the red laser. It is used to superimpose a near infrared laser and a red laser on the same axis. The galvanometer is used to move the scanning mirror in the vertical direction in order to oscillate the cold filter at a high speed and to vertically scan the laser beam. FIG. 6 shows a flowchart in which reflected light is imaged after the target fruit is irradiated with a laser beam and personal computer image processing is performed.

  FIG. 7 is a flowchart showing the signal flow of the two-wavelength three-dimensional visual sensor 4. Near-infrared and red laser beams are irradiated on the same axis, and the reflected light from the object is imaged on the PSD via the lens. By blinking the laser beam, it can be distinguished from sunlight, which is stationary light, and can be measured without being affected by sunlight. Thus, it can be used without distinction between day and night. Further, since PSDs having both near-infrared light and red light sensitivity were used, the near-infrared laser and the red laser were blinked at different frequencies to separate the near-infrared and red light-receiving components. Since the current obtained from the PSD is weak, it is amplified by an amplifier to convert the signal from current to voltage.

  FIG. 8 is a graph showing the spectral reflection characteristics of cherry tomatoes measured by the two-wavelength three-dimensional visual sensor 4. In this figure, red cram school fruits are shown with bold lines, and leaves and immature fruits are shown with other thin lines and dotted lines. FIG. 9 is a near-infrared image and shows the intensity of the near-infrared laser reflected light. FIG. 10 is a red image showing the intensity of the red laser reflected light. Moreover, FIG. 11 has shown the distance image to the position of the fruit measured with the galvanometer. FIG. 12 is a diagram in which a red image and a near-infrared image are compared for each pixel and a region is divided. Attraction strings etc. are also displayed in red. FIG. 13 is a diagram in which image processing is further performed and divided into the center of each fruit, the fruit stem, the foliage, and the like. In this figure, the fruit center is recognized by searching for the peak of the red image using the fact that the fruit surface is glossy and the reflected light from the center is strong. The 3D position of the fruit is calculated together with the distance image.

Next, the configuration of the fruit harvesting end effector 6 of the present invention will be described.
FIG. 14 is a side view showing the configuration of the fruit harvesting end effector according to the present invention, which is attached to the tip of the arm 5 of the manipulator 3. This fruit harvesting end effector 6 is attached to the tip from here, and a U-shaped metal fitting 11 and a chute 12 to take in to harvest the target fruit, a fruit storage box 13 for taking in the fruit discharged from the chute 12, A pedicle infuser 14 that holds down the pedicle W2 of the fruit contained in the U-shaped bracket 11, and a motor M3 that opens and closes the mouth of the U-shaped bracket 11 with a link mechanism. The motor M2 for twisting the U-shaped bracket 11 and the chute 12 (left and right rotation of the wrist), and the motor M1 for swinging the motor M2, the motor M3, the U-shaped bracket 11 and the chute 12 left and right (left and right swing of the wrist) It is configured. In this case, although not described in detail, the motor M1 is provided with a power transmission mechanism that swings the motor M2 to the left and right via the base 15. In addition, the pedicle infuser retainer 14 is preferably an elastic material. Accordingly, the frame portion of the motor M2 freely rotates on the base 15. Each motor uses a servo motor.

  FIG. 15 is a diagram of the fruit harvesting end effector 6 in a state in which the fruit pedicle W2 of the fruit taken into the U-shaped bracket 11 by the pedicle presser 14 is cut. The fruit taken in the U-shaped bracket 11 and the chute 12 is pressed between the U-shaped bracket 11 and the pedicle infuser 14 with its small berries W2. In this state, when the motor M2 is used for twisting a little, or when the motor M1 is shaken to the left or right, the fruit berry W2 is torn off and the fruit can be harvested in a state of having a wrinkle. In addition, 16 is a cam for opening and closing the peduncle retainer 14 in the figure indicated by a dotted line in the motor M2 portion.

  FIG. 16 (A) is a plan view showing the configuration of the fruit harvesting end effector 6 of the present invention, and FIG. 16 (B) is a partial detailed plan view. As described above, the motor M1 is driven so as to swing the motor M2, the U-shaped bracket 11 and the peduncle 14 to the left and right, and the motor M2 twists the U-shaped bracket 11 and the chute 12 (the left and right sides of the wrist). The motor M3 opens or closes the U-shaped bracket 11 by using the link mechanism and the cam 16 (see FIG. 14 and FIG. 15) to open the capillar presser 14 to cover the fruit. It is driven to close so as to press the pedicle W2. In this case, the wire 17 is connected between the front end portion of the pedicle infuser presser 14 and the fixing portion 18 of the motor M3. Bend in a concave shape. Then, when the fruit M is raised by the motor M3 and the fruit W enters the mouth of the U-shaped bracket 11, the fruit is more likely to enter the U-shaped bracket 11.

FIG. 17 is a diagram showing each posture when fruits are harvested by the fruit harvesting end effector 6 of the present invention. FIG. 17A is a diagram illustrating a left inclined posture. The left inclination is a posture in which the fruit harvesting end effector 6 is shaken by 30 ° to the right as shown in FIG. The fruit W targeted by this robot is attracted obliquely as shown in FIG. 1, and the distance to the main stem on the left side of the fruit bunch is short. For this reason, since the end effector 6 cannot be shaken greatly, the fruit to be harvested from the left has a left inclined posture. FIG. 17B is a view showing a posture in which the end effector 6 is straightened and harvested. FIG. 17C is a diagram illustrating a posture in which the end effector 6 is swung 90 degrees left and harvested from the right. FIG. 17D is a diagram illustrating a posture in which the end effector 6 is swung to the left by 110 ° and is tilted by 45 ° for harvesting. Mainly, the posture is to harvest the fruit on the other side of the bunch. On the left, right, and right slopes, if you take the path approaching the fruit W with the posture shown in the figure, you will hit an obstacle on the way, so when approaching the fruit, straighten the end effector 6 and approach the fruit. Shake right or left.
FIG. 18 is a diagram illustrating the relationship between the inclinations (A) to (D) when the fruit W is harvested.

One of the four harvesting paths is selected when the end effector 6 takes each path, and whether or not the obstacle is in a position where it hits the end effector 6 is determined as follows. Find out by counting.
(1) Counting the number of pixels in the end effector The three-dimensional position of each pixel in the three-dimensional image is examined, and the pixel at the position where the end effector 6 collides is counted. Main stems, leaves, immature fruits, fruit bells, small fruit bells, etc. are counted.
(2) Count the inflection at the position where it hits the U-shaped bracket 11.
Count the pixels of pericarp around the fruit.
(3) Are there other red immature fruits on the end effector pathway?
By the image processing, the above-described (1) to (3) were performed to determine the harvest route. At that time, since the U-shaped metal fitting 11 pushes the fruit stem when it approaches the fruit from the fruit stem side, it cannot be harvested, so the number of pixels counted in (2) is weighted and the number of pixels in (1) and (2) The route was selected so that there are few red-ripe fruits examined in (3).
FIG. 19 is a flowchart for selecting a route for performing the above operation.

Next, the mode in the case of tearing off the small fruit will be described.
FIG. 20 is a diagram in the case of harvesting by pulling a small peduncle of the fruit taken into the U-shaped metal fitting 11. In this case, the fruit is pressed between the U-shaped metal fittings 11 with the pedicle infuser 14 and pulled to be separated by delamination.
FIG. 21 is a diagram showing a case where bending is applied to the peduncle and separated by delamination. In this case, the small peduncle W2 is vertically held at the end of the U-shaped metal fitting 11, and is bent and separated by delamination.
FIG. 22 is a diagram showing a case where bending is also applied and separation is performed by delamination. In this case, the small peduncle W2 is pressed between the U-shaped metal fitting 22 and the small pedicle infuser 14 in a state of being bent, and in this state, the bending is applied and separated by delamination.

DESCRIPTION OF SYMBOLS 1 Robot 2 Robot base 3 Manipulator 4 3D visual sensor 5 Arm 6 End effector 7 CPU (computer)
8 TV monitor 11 U-shaped bracket 12 Chute 14 Small pedicel presser 16 Cam M1 First drive means M2 Second drive means M3 Third drive means W Fruit W2

Claims (3)

A traveling unit, a manipulator having six degrees of freedom of turning left and right, moving up and down, turning up and down, swinging the end effector (wrist) and rotating the wrist, a three-dimensional visual sensor, and a central processing unit are provided. In the end effector for harvesting of the robot, which is provided with arms on the manipulator so that it can be extended and retracted, and the fruit is taken in at the tip part.
A U-shaped bracket and chute that are taken in to harvest the fruit at the tip of the arm of the manipulator, a pedicle infuser that holds down the peduncle of the fruit captured in the U-shaped bracket, Third driving means for opening and closing the mouth portion of the U-shaped bracket with a link mechanism, second driving means for twisting the U-shaped bracket and the chute (wrist left and right rotation), the two driving means, U An end effector for fruit harvesting, comprising: a first bracket for swinging a letter-shaped bracket and a chute left and right (wrist swinging of the wrist).   Between the first driving means and the second driving means, there is provided a power transmission mechanism for swinging the U-shaped bracket, the pedicel infuser and the second driving means to the left and right. The end effector for fruit harvesting according to claim 1 characterized by things.   The harvesting path of the end effector is based on the number of three-dimensional pixels such as the main stem, leaves, immature fruit, fruit and small fruit in the three-dimensional visual sensor, and on the fruit and fruit in the position corresponding to the U-shaped bracket. 2. The fruit harvesting end effector according to claim 1, wherein the number of pixels of the other red immature fruit is determined by counting.