JP7285466B2 - harvesting robot - Google Patents

Description

The present invention relates to a robot that performs a fruit harvesting operation in a harvesting robot that automatically harvests fruits such as tomatoes and apples.

As a future social issue in Japanese society, there is concern about a decrease in the working population due to aging. In particular, agriculture, which is the primary industry, is in a very severe situation in terms of the aging population and labor shortage. According to the basic data of the Ministry of Agriculture, Forestry and Fisheries, the number of core farmers in 2014 was 1.68 million, a decrease of about 230,000 in five years. In addition, the average age is 66.5 years old, and the number of new farmers is also declining, which highlights the labor shortage due to the aging population. As a result, the amount of abandoned farmland reaches 400,000 ha, and this situation adversely affects the farming environment and living environment of the region. Especially in rural areas where there is a lot of arable land, depopulation is progressing due to the declining birthrate and aging population, and this problem is becoming more apparent because there are no people to bear the burden of agriculture.

Under such circumstances, expectations are rising for the automation of agriculture to deal with labor shortages. According to the Ministry of Economy, Trade and Industry's 2012 Robot Industry Market Trends, the domestic market for agricultural robots is expected to grow significantly from 2018 to 2024, and is expected to reach a scale of approximately 220 billion yen. In fact, developments that lead to the automation of agriculture, such as management using drones, automatic driving of tractors, and cultivation navigation systems, are becoming active. In addition, the Ministry of Agriculture, Forestry and Fisheries has a substantial subsidy system that supports the automation of agriculture, and the government has very high expectations for this field. Among them, with regard to technologies related to harvesting automation, companies, universities, etc. are developing robots for harvesting tomatoes, asparagus, and strawberries for practical use.

A number of patent documents have been published regarding technologies related to such automation of harvesting, such as sensing technology for finding fruits, technology for moving mechanisms, and technology for end effectors that perform picking motions. Among them, there is a conventional technique related to the end effector, such as that described in Patent Document 1. This is a useful method when the fruit is harvested by a method called delamination.

In the end effector in FIG. 1, 201 is a pull-in guide, 202 is a harvesting ring, 203 is a harvested fruit guide, 204 is a harvested fruit sheet, and 205 is a robot arm. The method of ripping off with this end effector is as follows. First, in the tomato pull-in operation, the pull-in guide 201 pulls in the harvested fruits to form a gap between the fruits. Harvest rings 202 are then inserted into the gaps between the fruits. As for the pull-in guide 201, the peduncle slips through the cut and is separated. The harvest ring 202 is then opened to pull the fruit and push the stem to elongate the stems and separate the fruit by delamination.

JP 2017-051104 A

However, in the above-described conventional configuration, there is a problem that the relative positional movement of each of the double harvesting rings is restricted by a predetermined drive mechanism, so that it is not possible to cope with various fruit situations. As a result, only the tearing operation can be performed by the operation of the end effector. Since the fruit can only be peeled off, it is difficult to harvest the fruit without a part called delamination, which is a factor in lowering the harvest rate. Furthermore, another problem is that the end effector itself becomes large and complicated due to having a plurality of components such as motors and guides, and cannot fit into the gaps between the branches. This is also a factor in lowering the yield.

Accordingly, an object of the present disclosure is to provide a harvesting robot capable of improving harvesting efficiency.

A harvesting robot according to the present disclosure includes:
a first robotic arm having a harvesting ring through which the object is passed;
a second robot arm having an auxiliary member for fixing a fruit peduncle bearing the object;
a control unit that controls the first robot arm and the second robot arm;
with
The control unit is configured to be capable of executing movement control and posture control of the harvesting ring while the fruit stem is fixed by the auxiliary member ,
The control unit is configured to be capable of executing a first operation of tilting the harvesting ring in the front-rear direction with respect to a horizontal plane so that the tip of the harvesting ring faces downward,
The control unit is configured to be capable of executing a second action of moving the harvesting ring upward while maintaining the posture of the harvesting ring after executing the first action,
After executing the second action, the control unit is configured to be capable of executing a third action of rotating the harvesting ring in a lateral direction with the tip of the harvesting ring as a starting point.
A harvesting robot.

According to the harvesting robot of the present invention, it is possible to stably harvest fruits, and it is possible to improve the harvesting efficiency of fruits.

A diagram showing a conventional robot hand described in Patent Document 1 A diagram showing the steps of a series of operations for harvesting according to the embodiment of the present invention. A diagram showing a tomato bunch to be harvested in the embodiment of the present invention. A diagram showing the steps of a tear-off operation according to the embodiment of the present invention. FIG. 4 is a diagram showing the positional relationship between the robot hand and the fruit to be harvested during the picking operation according to the embodiment of the present invention; FIG. 4 is a diagram for explaining a tear-off operation according to the embodiment of the present invention; FIG. 4 is a diagram for explaining a tear-off operation according to the embodiment of the present invention; FIG. 4 is a diagram for explaining a tear-off operation according to the embodiment of the present invention; FIG. 4 is a diagram for explaining a tear-off operation according to the embodiment of the present invention; FIG. 4 is a diagram for explaining a tear-off operation according to the embodiment of the present invention; FIG. 4 is a diagram for explaining a tear-off operation according to the embodiment of the present invention; 1 is a top view of a harvest ring according to an embodiment of the invention; FIG. A diagram for explaining the device configuration of the harvesting robot in this embodiment.

First, in order to facilitate understanding of the method for estimating the position of fruit stems in the present embodiment, (1) an example form of a tomato to be harvested, and (2) the case where the automatic harvesting robot in the present embodiment is used An example form of a basic harvesting operation will be described.

(1) Morphology of Tomatoes to be Harvested In the description of the present disclosure, the tomato shown in FIG. 3 will be described as an object to be harvested. As shown in FIG. 3, one bunch is constructed around a stem called a fruit stem 405 . From this fruit stem 405, a small fruit stem 401 and a sepal 403 are grown, and a fruit 404 is grown therefrom. There is a portion called delamination 402 in the middle of the peduncle 401 , and when a force is applied to the peduncle 401 , the peduncle 401 is divided at the delamination 402 . In the present disclosure, the target for harvesting is not a cluster composed of a plurality of or more fruits 404, but each fruit 404 is targeted for harvesting, and each fruit 404 is harvested by tearing off each fruit 404. are doing. This is because it is assumed that the degree of maturity of the fruit 404 attached to the bunch is different. In general, cherry tomatoes are eaten raw and must be harvested together with the sepals 403 without damaging the surface of the fruit 404 .

(2) Form of Harvesting Operation FIG. 13 shows the device configuration of the harvesting robot in this embodiment.

The image acquisition unit 1401 is an image sensor, and captures an image of an area where the fruit 404 to be harvested exists. An image processing unit 1404 performs image processing on the image captured by the image acquisition unit 1401 . A determination unit 1405 detects a tomato fruit to be harvested based on the result of image processing. The control unit 1406 controls the first robot arm 1402 based on the detection result by the determination unit 1405 to move the tomato harvesting robot to a position suitable for harvesting. After that, the image acquisition unit 1401 further performs imaging, and the image processing unit 1404 performs processing, whereby the determination unit 1405 identifies the three-dimensional position, color, and the like of the tomato fruit. Then, the control unit 1406 controls the robot arm 1402 with the harvesting ring and the robot arm 1403 with the auxiliary member to harvest tomatoes. The above is the basic device configuration and its operation.

Hereinafter, the robot arm 1402 with the harvesting ring will be referred to as "first robot arm 1402", and the robot arm 1403 with the auxiliary member will be referred to as "second robot arm 1403".

Embodiments of the present disclosure will be described below with reference to the drawings.

(Embodiment)
FIG. 2 shows an example of basic operation steps of a harvesting operation by the automatic harvesting robot of the present disclosure.

First, clusters on which fruits to be harvested are grown are detected (STEP 1). Regarding the detection method, there are detection by applying image processing to the image acquired using the image sensor, detection by learning the image of the bunch where the fruit to be harvested is growing by artificial intelligence learning, etc. Conceivable.

Next, the image acquisition unit 1401 performs a sensing operation for harvesting the fruit on the bunches detected in STEP 1 (STEP 2). In order to harvest the fruit, it is necessary to know whether the fruit is ready for harvesting (whether it is ripe), the position of the fruit, and the positions of the stems and leaves around the fruit. This is an operation for calculating the information of Here, as described above, in order to calculate the color and position of the fruit, sensors used for sensing include a color camera for acquiring a color image, a distance sensor capable of measuring the distance, or a sensor for measuring the distance to the object. A method of simultaneously using a TOF camera or the like capable of measurement is adopted. Fruit color is used to determine whether the fruit is ripe.

Next, based on the sensing data acquired in STEP 2, the determination unit 1405 calculates the color of the fruit and the positions of the fruit and surrounding stems and leaves (STEP 3).

Next, the determination unit 1405 performs a trajectory calculation as to how the first robot arm 1402 should be extended in order to harvest the fruit determined to be harvested (STEP 4). At this time, it is important to calculate a trajectory in which the first robot arm 1402 does not interfere with the surrounding stems and leaves from the position information.

Next, based on the trajectory calculated in STEP 4, the first robot arm 1402 operates to bring the robot hand closer to the fruit to be harvested (STEP 5).

Finally, the robot hand performs a stripping operation (STEP 6).

The robot hand according to this embodiment will be described in detail below. FIG. 6 shows a schematic perspective view of the robot hand in this embodiment.

The robot hand 700 in this embodiment is mounted on the first robot arm 1402 . The fruit picking operation by the robot hand 700 is realized by the control of the first robot arm 1402 by the control unit 1406 .

The robot hand 700 includes a harvesting ring 701 and a connecting portion 703 that connects the harvesting ring 701 and the first robot arm 1402 . The robot hand 700 may be configured by omitting the connecting portion 703 and directly attaching the harvesting ring 701 to the first robot arm 1402 .

Here, the harvesting ring 701 will be described in detail with reference to FIG. 12 . FIG. 12 is a schematic top view of the harvesting ring in this embodiment. 12 corresponds to the first robot arm 1402 side in FIG.

The harvesting ring 701 forms a ring hole with a square tip. Specifically, the harvesting ring 701 has a first bent portion 1301 at its tip. The harvesting ring 701 has a first straight portion 1302 and a first straight portion 1303 on the left and right so as to form a first bent portion 1301 (that is, square shape). Harvesting ring 701 also has a second straight portion 1306 and a second straight portion 1307 that are connected to connecting portion 703 in FIG. A second bent portion 1304 is formed by the first straight portion 1302 and the second straight portion 1306 . A second bent portion 1305 is formed by the first straight portion 1303 and the second straight portion 1307 .

The first bent portion 1301 is preferably at an angle of 90°. This is due to the fact that the fruits to be harvested are spherical. When the first bent portion 1301 approaches an acute angle from 90°, when the harvesting ring 701 is inserted, the first bent portion 1301 collides with the stem before the fruit to be harvested passes through the harvesting ring 701. more likely. In particular, when the angle of the first bent portion 1301 is smaller than 70°, the probability of collision increases. This makes it difficult to carry out stable harvesting. In addition, when the tip portion approaches an obtuse angle from 90°, the size becomes larger than the size necessary for passing the harvesting ring 701, and interference occurs between the harvesting ring 701 and the stems, leaves, etc. at the time of harvesting. It's easy to do. In particular, when the angle of the first bent portion 1301 becomes larger than 130 degrees, the probability of interference increases. This makes it difficult to carry out stable harvesting. From the above, the angle of the first bent portion 1301 is desirably 70° or more and 130° or less, and particularly desirably 90°.

Harvesting ring 701 also allows fruit to be harvested to pass through it, as will be described later. Therefore, it is desirable that the distance between the second straight portion 1306 and the second straight portion 1307 is larger than the maximum diameter of the fruit. However, if this harvesting ring 701 is large, it becomes difficult to pass between the branches when harvesting the fruit. For this reason, it is preferable that the size of the harvesting ring 701 itself is as small as possible.

The stripping operation of STEP6 in FIG. 2 will be described in detail below. FIG. 4 shows the operation steps of the robot hand in this embodiment. FIG. 5 shows a schematic diagram of the positional relationship of the robot hand with respect to the fruit to be harvested when performing the operation. Further, FIGS. 7 to 11 show schematic perspective views of the movement of each step in the tearing operation.

A robot hand 700 for the tearing operation is attached to the first robot arm 1402, and an auxiliary member 704 for creating a fulcrum for the tearing operation is attached to the second robot arm.

First, in STEP 1 in the action steps of FIG. 4, the control unit 1406 tilts the harvesting ring 701 by 20° with respect to the horizontal plane by controlling the first robot arm 1402 (corresponding to the "first action" of the present invention). . Specifically, the harvesting ring 701 is tilted 20° in the front-rear direction with respect to the horizontal plane so that the tip of the harvesting ring 701 faces downward. This is because when the harvesting ring 701 is inserted in the fruit stem 405 direction in STEP 2, it is possible to insert it smoothly by inserting it with an inclination. If the inclination is not this, when the position of the harvesting ring 701 is displaced vertically with respect to the lower end of the fruit 404 in STEP 2, the resistance when colliding with the fruit 404 increases, and the possibility that it cannot be inserted well increases. By providing an angle of 20°, it is possible to reduce the resistance from the fruit 404 other than the harvested object at the time of insertion. Also, this angle of 20° depends on the shape of the curved surface of the lower end of the tomato, but if it is in the range of 10° to 30°, generally smooth insertion is possible. In other words, the angle at which the harvesting ring 701 is tilted should be 10° or more and 30° or less with respect to the horizontal plane of the ring surface of the harvesting ring 701 .

Next, in operation STEP2 in FIG. 4, the control unit 1406 inserts the harvesting ring 701 in the fruit stem 405 direction. Specifically, as shown in FIG. 7, the harvesting ring 701 is inserted in the direction of the stem 405 so that the tip of the harvesting ring 701 contacts the lower end of the fruit 404 to be harvested. At this time, as described above, in STEP 1, it is inserted while being tilted by 20°. This makes it easy to insert smoothly.

In STEP 2, the control unit 1406 needs to detect the position of the lower end of the fruit 404 to be harvested. At this time, it is possible to detect the position by, for example, a TOF camera mounted on the first robot arm 1402, but the measurement accuracy is improved by performing sensing as close to the fruit 404 as possible. Further, for example, when sensing is performed from the position of the harvesting robot main body, it is conceivable that the fruit 404 to be harvested may be hidden by stems, leaves, etc., and cannot be measured. Therefore, it is desirable to perform sensing as close to fruit 404 as possible. Therefore, it is preferable to attach a sensor such as a TOF camera near the tip of first robot arm 1402 in order to detect the position of the lower end of fruit 404 .

A method for detecting the position of the lower end of fruit 404 by the sensor attached to first robot arm 1402 will be described. This method is performed prior to operation STEP2 in FIG.

First, a sensor attached to the first robot arm 1402 acquires an image of the fruit 404 to be harvested from an arbitrary position, preferably near the body of the harvesting robot. Let the imaging position at this time be a 1st imaging position. Then, image processing is performed on the image acquired at the first imaging position, and a position suitable for detecting the lower end of the fruit to be harvested, specifically, a position as close as possible without any obstacles to the imaging, is selected as the second imaging position. 2 imaging position. At the second imaging position, image processing is performed on the image information at the first imaging position to identify the position of the fruit 404 and the positions of the surrounding main stem and fruit stem. Calculated. Also, the second imaging position is preferably a position where no obstacle exists between the sensor attached to the first robot arm 1402 and the fruit 404 .

Next, the control unit 1406 controls the first robot arm 1402 to move the sensor attached to the tip of the first robot arm 1402 to the second imaging position, and performs imaging at the second imaging position. . Also in this case, in order to measure the position of the lower end of the fruit, it is necessary to use a color image and a sensor that simultaneously measures the distance to the object, and a TOF camera or the like is used.

With the above method, the position of the lower end of fruit 404 can be detected with higher accuracy.

Next, in operation STEP 3 in FIG. 4 , the control unit 1406 controls the second robot arm 1403 to press the auxiliary member 704 against the stem 405 . Specifically, as shown in FIG. 8, the auxiliary member 704 is pressed against the stem 405 located above the fruit 404 to be harvested from the same direction as the insertion direction of the harvesting ring 701 . This allows the harvesting ring 701 to perform a stable harvesting operation. The operation of STEP3 may be performed simultaneously with the operation of STEP2, or may be performed prior to STEP2.

In STEP 3 , the control section 1406 needs to detect the position of the fruit stem 405 . By attaching a sensor for detecting the stem 405 to the tip of the second robot arm 1403, the position of the stem to be pressed can be detected with high accuracy. As for this sensor, it is necessary to use a sensor that simultaneously measures the color image and the distance to the object, and a TOF camera or the like is used.

Next, in operation STEP4 in FIG. 4, the control unit 1406 controls the first robot arm 1402 to push up the harvest ring 701 along the side of the fruit 404 (corresponding to the "second operation" of the present invention). Specifically, as shown in FIG. 9, the harvesting ring 701 is pushed up so that the tip of the harvesting ring 701 moves along the side of the fruit 404 . Here, since the tip of the harvesting ring 701 is along the side surface of the fruit 404 , the harvesting ring 701 can be smoothly moved to the upper end of the fruit 404 . Here, if the harvesting ring 701 is pushed up in a direction away from the fruit 404 instead of along the side of the fruit 404, the harvesting ring 701 is likely to come into contact with the stem 405, and the harvesting ring 701 is pushed up by resistance from the stem 405 or the like. may not be available.

In STEP 4, it is desirable that the distance by which the harvesting ring 701 is pushed up corresponds to the length of the fruit 404 in the vertical direction. The length in the vertical direction can also be calculated with high accuracy by using the data acquired at the second imaging position.

Further, in operation STEP5 of FIG. 4, the control unit 1406 controls the first robot arm 1402 to twist the harvesting ring 701 while pushing up the harvesting ring 701 (the "third arm" of the present invention). (equivalent to "action"). Specifically, as shown in FIG. 10, the tip of the harvesting ring 701 is used as a starting point, and the harvesting ring 701 is twisted so as to rotate in the horizontal direction. This makes it possible to suppress the resistance to the harvesting ring 701 from the stem 405 and the like during the push-up operation. Also, the degree of rotation in the twisting motion may be linearly proportional to the push-up motion in STEP 4 of FIG. 4, or may increase exponentially. The operation of STEP5 is performed in parallel with the operation of STEP4.

Further, in operation STEP6 of FIG. 4, the control unit 1406 draws the harvesting ring 701 by controlling the first robot arm 1402 (corresponding to the "fourth operation" of the present invention). Specifically, as shown in FIG. 11, the harvesting ring 701 is pulled toward the first robot arm 1402, that is, in the direction opposite to the insertion direction. That is, harvest ring 701 moves away from fruit 404 . At this time, the auxiliary member 704 pressed in STEP 3 serves as a fulcrum, and the tip of the harvesting ring 701 serves as a point of action, so that a shearing force is generated in the vicinity of the delamination portion of the fruit to be harvested, and the fruit 404 becomes a small peduncle. 405. The fruit 404 separated from the peduncle 405 is stored, for example, in a harvesting basket (not shown) arranged below the harvesting ring 701 .

The above operational steps enable the fruit to be picked.

[effect]
As described above, the harvesting robot according to this embodiment includes the first robot arm 1402 having the harvesting ring 701 through which the fruit (object) 404 is inserted, and the auxiliary member 704 for fixing the stem 405 bearing the fruit 404. and a control unit 1406 that controls the first robot arm 1402 and the second robot arm 1403, and the control unit 1406 fixes the stem 405 with the auxiliary member 704, It is configured to be able to execute movement control and attitude control of the harvesting ring 701 .

Therefore, according to the harvesting robot according to the present embodiment, it is possible to harvest fruits without using a plurality of rings as in the prior art. As a result, it is possible to easily harvest the fruit present in the gaps between the branches, and also to harvest the fruit without delamination, so that the fruit can be harvested efficiently.

The robot arm mounted on the harvesting robot of the present invention is characterized by having a harvesting ring suitable for harvesting fruits. Applicable in general.

201 Pull-in guide 202 Harvesting ring 203 Harvested fruit guide 204 Harvested fruit sheet 205 Robot arm 401 Small fruit stem 402 Delamination 403 Sepal 404 Fruit 405 Fruit stem 700 Robot hand 701 Harvesting ring 703 Connecting part 704 Auxiliary member 1301 First bending part 1302, 1303 First straight portion 1304, 1305 Second bent portion 1306, 1307 Second straight portion 1401 Image acquisition unit 1402 Robot arm with harvesting ring 1403 Robot arm with auxiliary member 1404 Image processing unit 1405 Judging unit 1406 Control unit

Claims (6)

a first robotic arm having a harvesting ring through which the object is passed;
a second robot arm having an auxiliary member for fixing the fruit peduncle of the object;
a control unit that controls the first robot arm and the second robot arm;
with
The control unit is configured to be capable of executing movement control and posture control of the harvesting ring while the fruit stem is fixed by the auxiliary member ,
The control unit is configured to be capable of executing a first operation of tilting the harvesting ring in the front-rear direction with respect to a horizontal plane so that the tip of the harvesting ring faces downward,
The control unit is configured to be capable of executing a second action of moving the harvesting ring upward while maintaining the posture of the harvesting ring after executing the first action,
After executing the second action, the control unit is configured to be capable of executing a third action of rotating the harvesting ring in a lateral direction with the tip of the harvesting ring as a starting point.
harvesting robot. The harvesting ring forms a ring hole with a square tip.
A harvesting robot according to claim 1 . The harvesting ring includes a pair of first linear portions connected to each other at one end thereof and the other end of the pair of first linear portions connected to the other end of the pair of first linear portions so as to form the angular shape, and the first robot arm is connected in parallel to each other. and a pair of second straight portions extending to the proximal side of the
A harvesting robot according to claim 2. The control unit performs the first operation so that the ring surface of the harvesting ring forms an angle of 10° or more and 30° or less with respect to a horizontal plane.
A harvesting robot according to claim 1 . The control unit is configured to be capable of executing a fourth operation of moving the harvesting ring in a pulling direction after executing the third operation.
A harvesting robot according to claim 1 . The control unit, when harvesting the target object,
disposing the harvesting ring below the object in the first operation;
inserting the object into the harvesting ring and moving the harvesting ring to an upper end of the object in the second operation;
In the third operation and the fourth operation, the harvesting ring generates a shearing force on the stem existing at the upper end of the object to separate the object from the stem.
A harvesting robot according to claim 5 .

Images