JPS62275611A - Guide apparatus of working machine for fruit vegetables - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] 3. Detailed Description of the Invention [Field of Industrial Application] The present invention is directed to fruit vegetables used for growing management and harvesting of fruits such as mandarin oranges and apples, and fruit vegetables such as tomatoes. Regarding a guidance device for a work machine for fruits and vegetables for guiding a work hand of a work machine towards a work target, in detail, the guidance device for a work machine for fruits and vegetables is configured to guide a work hand of a work machine towards a work target. It is equipped with a visual device that detects a highlight region that appears brightest in a region that corresponds only to color, and a geometric feature point of the region, and that detects when the target fruits and vegetables are approached within a set distance. guiding means for guiding a working hand equipped with a detecting proximity sensor in the direction of the geometric feature point until the proximity sensor is activated, and then guiding it in the direction of the highlight area; This invention relates to a guidance device for a working machine for fruits and vegetables.

[Conventional technology]

The above-mentioned guidance device for this type of fruit and vegetable work machine is explained using fruits such as tangerines and apples as an example.These fruits have different colors depending on the texture of the fruit tree, leaves, or background objects such as the sky. Therefore, by extracting a specific color area corresponding to the color of the work target from the image information of the work target,
Being able to separate and identify the work target and other objects,
Since there is a position corresponding to each fruit where light is reflected strongly on the surface of the fruit, by extracting the highlight area that appears brightest within the above specific color area, it can be determined that Based on the fact that it can be used as position information corresponding to each work object, it is possible to easily guide the work object to its location without measuring the distance to the work object.

In order to efficiently perform a predetermined task such as harvesting on each of a plurality of work objects in the same area, the applicant has developed a system in which the work hand is placed at a predetermined distance from the work object. Detects when the work hand approaches the work object within a set distance so that the work hand can move at high speed until it approaches the work object, and after that, the work hand can reliably capture the work object. The working hand is equipped with a proximity sensor, and until the proximity sensor is activated, the geometric feature point of the region of the specific color is a position where the work object is definitely present on the front side of the working hand; for example,
We have proposed a means for guiding the object toward the highlight area by moving it at high speed toward the center of gravity, etc., and moving it at low speed by the operating distance of the proximity sensor as the proximity sensor operates.

[Problem that the invention seeks to solve]

However, the means proposed for the above-mentioned light had the following disadvantages, and there was room for improvement.

In other words, a region of a specific color corresponding to the color of the work target may be seen as a single region due to overlapping of multiple work targets, but for example, if one work target fruit or vegetable is
In some cases, it may appear deformed due to the shadow of other objects such as leaves or other objects around it, so the position of the geometric feature point in the above specific color area can be used to capture one target fruit or vegetable. It may be in a certain position. In addition, the highlight area that appears brightest within the above specific color area may be due to the fact that the surface of the work target is not a perfectly smooth surface or the influence of reflected light from other work targets or surfaces such as leaves in the surrounding area. , there may be multiple objects for one work object, or they may appear on the edge of the above-mentioned specific color area, so if the work hand is approached only in the direction where the above-mentioned highlight area is located, There was a risk that the system would fail to capture the work target, resulting in a decrease in work efficiency.

The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to make it possible to efficiently guide a working hand in the direction in which fruits and vegetables to be worked are located.

[Means for solving problems]

The characteristic configuration of the guidance device for a working machine for fruits and vegetables according to the present invention is as follows:
The guiding means moves the working hand by the set distance in the direction of the geometric feature point even after the proximity sensor is activated, and captures the fruits and vegetables to be worked by the movement. The working hand is configured to be guided in the direction of the highlighted area only when it is not possible to do so, and its operation and effects are as follows.

[For production]

In other words, even after the proximity sensor is activated, the work hand can be guided toward the geometric feature points of the area corresponding to the color of the fruits and vegetables to be worked on, such as the center of gravity or equidistant points of the area. , it checks whether or not it can capture the work target, and only if it cannot capture it, guides it in the direction of the highlighted area.

〔Effect of the invention〕

Therefore, if the work hand can be guided toward the geometric feature point and the work target can be captured, there is no need to perform the process of guiding the work hand in the direction of the highlight area. It has now become possible to improve work efficiency for target fruits and vegetables.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in Fig. 6, a boom (2) having a dogleg shape in side view is attached to a vehicle body (V) equipped with a pair of left and right wheels (1), (1) at the front and rear, so that it can be raised and lowered as well as rotated. An auxiliary boom (3) that can swing horizontally is attached to the tip of the auxiliary boom (3), and a manipulator (4) for work is attached to the tip of the auxiliary boom (3) so as to be able to rotate. This is a working machine for fruit harvesting. In Fig. 6, (5) is the hydraulic cylinder for raising and lowering the boom, (
6) is an electric motor for swinging the boom, and (7) is an electric motor for swinging the auxiliary boom.

The manipulator (4) has a multi-jointed telescopic arm (8).
) and a work hand () 1) for fruit harvesting attached to the tip of the telescoping arm (8), and the telescoping arm (8) is rotated by an electric motor (9a) to ), and is structured so that it can be rotated around the horizontal axis (Y) by an electric motor (9b), and can be expanded and contracted by an electric motor (9c) for expansion and contraction. There is.

A color video camera (S0) constituting the visual device (A) for detecting the position of the target fruit (F) is connected to the vertical axis (X) and horizontal axis (Y) of the telescopic arm (8). The field of view is stored at the intersection with the direction of the working hand (H) pointing.

To explain the working hand (H), as shown in FIGS. 4 and 5, it has a base frame (10
) is attached to the tip of the telescoping arm (8), and a trap case (II) is attached to the tip of the base frame (10).
) is installed.

In addition, a ventilation pipe (12) having a soft rubber vacuum pad (13) on the outer surface thereof is attached to the catching part case (II).
), and a branch pipe (14) for ventilation is provided on the outer surface of the base of the ventilation pipe (12), and the branch pipe (14) A telescopic bellows type hose (15) is connected to. Then, as shown in FIG. 1, the bellows type hose (15)
is connected to the intake pump (16) by piping. Further, a negative pressure sensor (S1) for detecting that the fruit to be worked on (F) is adsorbed to the vacuum pad (13) is attached to the branch pipe (14), and a negative pressure sensor (S1) is attached to the branch pipe (14). An infrared emitting type proximity sensor (S2) that non-contact detects when the target fruit (F) has approached within a set distance from the target fruit (F), the detection action part of which is connected to the vacuum pad (
13) with the distal end facing forward. Furthermore, the first
In the figure, (29) is an electromagnetically operated on-off valve that controls suction and stop of the vacuum pad (13).

To explain the structure of the proximity sensor (Sz), a sensor main body (17) housing a light emitting part, a light receiving part, etc. is held in the branch pipe (14), and light is emitted and received by the sensor main body (17). Optical fiber (1
8) extends into the inside of the vacuum pad (13) through the inside of the ventilation pipe (12), and its extended end is supported at the center of the vacuum pad (13) by a holder (19). be.

Furthermore, a camera (20a) for cutting the fruit stalk of the fruit captured while being attracted to the vacuum pad (13) and a device for pressing and supporting the fruit stalk are installed at the tip of the trapping part case (II). A stalk cutting device (20) consisting of a stalk pressing member (20b) is provided.

To explain the stalk cutting device (20), each of the force lid (20a) and the stalk pressing member (20b),
It is formed into a band-like semicircular arc shape, and is pivotally connected to the catching part case (II) so as to be freely rotatable around a coaxial center, and supports the stalk of the caught fruit between them. a state in which the catching part is protruded forward from the catching part case (II);
It is operatively connected to an electric motor (22) via a differential mechanism (21) so that it can be switched to a retracted state to form a passage for introducing fruit between them. In other words, when either the cutter (20a) or the stalk pressing member (20b) comes into contact with the stalk first, the raising operation of the one in contact is stopped, and the other continues to be raised up. It is made so that it can be moved. Further, a limit switch (S3) for detecting the completion of the raising operation of the fruit stem cutting device (20) and a glue mint switch (S4) for detecting the completion of the retiring operation are connected to the input gear (21) of the differential mechanism (21).
23) so that it can be operated by a protrusion (24) attached to it. In the figure, (24) is an air cylinder for cutting the power lid (20a), (25) is a protective cover to prevent damage to the fruit during approach, and (26)
) is an air cylinder for opening and closing, and (27) is an air cylinder for operating the vacuum pad (13).

Next, a configuration of a means for guiding the work hand (I) toward the work target fruit (F) based on image information captured by the camera (S0) will be described.

As shown in Fig. 1, by image processing the imaging information from the camera (S0), the brightest area (F0) corresponding only to the color of the work target fruit (F), which will be described in detail later, is Functions as a visual device (8) that detects the light area (P0) and the geometric feature points (P1) of the area (F0) corresponding only to the color of the work target fruit (F), and also provides image information thereof. and the proximity sensor (S2), negative pressure sensor (S1), and limit switch (h
), (S4), etc., and various information stored in advance, each air cylinder (24) operates the lifting cylinder (6), vacuum pad (13), and work hand (H). , (26), (27),
Electric motors for turning and swinging (6), (7)
, (9a), (9b), (9c), etc., constitutes a guiding means (B) that guides the working hand (H) toward the fruit to be worked on (F). A control device (28) is provided.

To explain image processing by the visual device (A), as shown in FIGS.
From the color image information (CO) (see Figure 3 (a)) of the fruit to be worked on (F) obtained in step 0), the fruit to be worked on (F) is
) Obtain binarized image information (CI) (see Fig. 3 ([1)]) in which the region (F0) corresponding only to the color of
Extracting the gray level image information (C) obtained by quantizing the gray level of the color image information (C0) (see FIG. 3 (c)),
This grayscale image information (C2) is converted into the binarized image information (C2).
1), mask the highlight area (P0) that looks brightest in the area (F0) corresponding only to the color of the fruit to be worked on (F), and only the color of the fruit to be worked on (F). Each center of gravity (P1) as a geometric feature point of the corresponding region (F0) is determined (see FIG. 3(d)).

In other words, the work target fruit (F) and the techniques and leaves around it,
Or because the background sky etc. has different colors,
If an area (F0) corresponding only to the color of the fruit to be worked on (F) is extracted from the color image information (C0) obtained by the camera (s0) and binarized, an area corresponding only to the fruit to be worked on (F) is obtained. In addition to being able to extract
By extracting the highlight region (P0) that looks the brightest based on the gray level of the grayscale image information (C2) in F0), the information on its position can be extracted from the 2 This takes advantage of the fact that the digitized image information (C1) can be used as information representative of the position of the fruit price within the recognized area (F0). Moreover, the center of gravity (p1) of the area (F0) corresponding only to the color of the work target fruit (F) is the highlight area (P
Since it is not always the case that there is only one place for each fruit, and there is no problem if there is a place near the center of the fruit, the working hand (2) is directly placed in the high place. If the guide is directed toward the light area (P0), there is a risk that the proximity sensor (S2) may not operate, so first, the guide is directed toward the center of gravity (P1) with the fruit definitely present in front. It is designed to guide you.

Next, based on the detection information of the highlight area (P0) and center of gravity (p1), the configuration of the guiding means (B) for guiding the working hand (2) toward the target fruit (F) is determined. , will be explained in detail while explaining the operation of the control device (28).

As shown in Fig. 2, by capturing an image of the range in which the work target fruit (F) is located using the camera (S0) and performing the above-mentioned image processing, the image corresponding only to the color of the work target fruit (F) is obtained. The center of gravity (P, ) of the area (F0) and the highlight area (P0) are detected, and the direction in which the work target fruit (F) is located relative to the current position of the work hand (H) is calculated. However, the center of gravity (p1) and the highlight area (
If no P0) is detected, it is determined that there is no fruit (F) to be worked on within the imaging range, and the next image processing within the imaging range is performed (step #1.
Step #2).

Next, the work target fruit (F) extracted by the image processing
The size and center of gravity (P1) of the area (F0) corresponding only to the color of
) and the number of highlight areas (P0) and their brightness, etc., determine the work order, and based on that order, the suction port of the vacuum pad (13) is connected to the extracted work target fruit (F). One area corresponding only to the color of (F
0) while maintaining a state facing the direction in which the center of gravity (P, ) is located, the telescopic arm (8) is protruded, and the working hand (H) is moved until the proximity sensor (S2) is activated. The robot is made to approach the center of gravity (P1). However, if the proximity sensor (S2) does not operate even if the approach distance exceeds a preset distance, the telescoping arm (8) is shortened and returned to the original approach start position to stop the approach. (Stephbu #
3~Stephbu! 16).

As the proximity sensor (S2) operates, the vacuum band (13) starts suction operation, until the negative pressure sensor (S1) operates and detects the adsorption of the fruit (P) to be worked on. - For example, the proximity sensor (S2)
Protrude forward in the same direction of movement by a set distance set based on the operating distance of. However, if the negative pressure sensor (S1) does not operate even after the working hand (H) moves the set distance, it is determined that the capture has failed, and after retreating by a distance corresponding to the set distance, Based on whether or not the negative pressure sensor (S1) operates again, it is determined whether or not the fruit to be worked on (F) has been sucked (Step #7 to Step #10).

When the negative pressure sensor (S1) does not operate in the suction tube #10 and detects a failure in fruit suction, the work hand (H)
The number of corrections is determined based on whether the number of repetitions of the highlight correction process that is guided by changing the direction of the highlight area (P0) reaches the number of extracted highlight correction processes (Pa). Check. If the number of corrections does not reach the set number, the above correction number? ? #
While performing the suction operation of the vacuum pad (13) from 7 onwards, the process of moving the highlight area (P0) in one direction is repeated until the negative pressure sensor (S,) is activated. However, the highlight area (P0
), the telescoping arm (8) is shortened and returned to the original approach starting position to stop the approach. ).

In this embodiment, in the highlight correction process in which the direction is changed and guided in the direction of the highlight correction process (P0), not only the direction of the extracted highlight area (P0) but also the center of gravity (P, ) For example, if the fruit is approached toward the highlight area (P0) of the rearmost fruit among multiple fruits overlapped in the front-rear direction,
Even in the event of a failed catch, the robot can automatically re-approach something in front of it.

When the negative pressure sensor (S,) operates and detects that the target fruit (F) has been suctioned as desired, the vacuum pad (13) is operated to retire, and at the same time, the work hand (H) is moved forward. moving it to the side and taking the adsorbed fruit into the operating range of the stem cutting device (20),
The fruit stem cutting device (20) is operated to cut the fruit stem. That is, as the negative pressure sensor (S,) operates, the cutter (20a) and the stalk pressing member (20b) are operated to raise, and the glue limit switch (S,) for detecting the completion of the raising operation is activated. Based on the detected information, it is determined whether or not the standing operation has been completed.

As the force lid (20a) and the keratin pressing member (20b) are raised up, the vacuum pad (1
After temporarily stopping the suction operation in step 3) and retracting the working hand (H) in this state to position the cutter (20a) on the base side of the fruit stem, the cutter (20a) is
Activate 0a) to cut the stem. Immediately after cutting, the suction operation of the vacuum pad (13) is restarted to suck the cut fruit again, and then the force lid (20a)
Then, the helical pressing member (20b) is operated to fall down until the limit switch (S4) for detecting the completion of the retraction operation is activated, the telescopic arm (8) is shortened, and the working hand () I) is approached. Returning to the starting position, protruding the vacuum pad (13), stopping the suction operation, and ejecting the harvested fruit, it is determined whether the harvesting operation has been performed normally. After releasing the harvested fruits, the number of approaches is checked based on the number of fruits in one screen recognized by the image processing to determine whether approaches to all fruits have been completed. The above-described operations are repeated until no fruit can be recognized within the image area (steps 113 to #18).

However, the glue used to detect the completion of the standing operation
3) does not operate and the rising operation is not completed, it is determined that there is a problem of failure in harvesting, such as other objects being caught, and the power lid (20a) and the stem are removed. The pressing member (20b) is operated to fall until the limit switch (S4) for detecting the completion of the retiring operation is operated, and
The telescoping arm (8) is shortened to form a working hand (H'
) to the approach start position and repeat step #3 above.
I am trying to restart the approach to one fruit from.

[Another example]

In the above embodiment, the case where the center of gravity (P1) of the area (F0) corresponding only to the color of the extracted fruits and vegetables is used as the geometric feature point, but for example, the extraction area (F0)
Other geometrical features may be used, such as points equidistant to the boundary of .

Further, in the above embodiment, the structure for harvesting fruits was exemplified, but the present invention can also be applied to other operations such as harvesting vegetables such as tomatoes.

To carry out the present invention: 1. The configuration of each part, such as the specific configuration of the working hand (H) and the specific configuration for guiding the working hand (H), can be changed in various ways depending on the work object and work form.

[Brief explanation of the drawing]

The drawings show an embodiment of the guidance device for a working machine for fruits and vegetables according to the present invention, FIG. 1 is a block diagram showing the configuration of the control system, FIG. 2 is a flowchart showing the operation of the control device, and FIG.
Figures (a) to (d) are explanatory diagrams of image processing by the visual device;
FIG. 4 is a cutaway plan view of the working hand, FIG. 5 is a cutaway side view thereof, and FIG. 6 is an overall side view of the fruit harvesting machine. (F)... Fruits and vegetables to be worked on, (F0)...
...An area corresponding only to the color of the fruits and vegetables to be worked on, (P0)
...Highlight area, (P1) ...Geometric feature point, (S2) ...Proximity sensor, (H
)...Working hand, (A)...Visual device, (B)...Guidance means.

Claims (3)

[Claims] (1) Based on the image information of the fruits and vegetables to be worked on (F), a highlight area (P
_0) and the geometric feature point (F_0) of the area (F_0)
A working hand (II) is equipped with a visual device (A) that detects P_1) and a proximity sensor (S_2) that detects that the work target fruits and vegetables (F) have approached within a set distance. , comprising a guiding means (B) for guiding toward the geometric feature point (P_1) until the proximity sensor (S_2) is activated, and then guiding toward the highlight area (P_0). A guidance device for a working machine for fruits and vegetables, wherein the guidance means (B) is connected to the proximity sensor (
Even after S_2) is activated, the working hand (
H) is the geometric feature point, and only when the fruit and vegetable to be worked on (F) cannot be captured due to its movement, the work hand (H) is directed in the direction of the highlight area (P_0). A guidance device for a fruit and vegetable work machine configured to provide guidance. (2) The work for fruits and vegetables according to claim 1, wherein the geometric feature point (P_1) is the center of gravity of a region (F_0) corresponding only to the color of the fruit and vegetables to be worked (F). The aircraft's guidance system. (3) The fruits and vegetables according to claim 1, wherein the geometric feature point (P_1) is an equidistant point of an area (F_0) corresponding only to the color of the fruit and vegetables to be worked on (F). Guidance device for work equipment.