JPS63102611A - Guide apparatus of working machine for fruits and vegetables - Google Patents

Abstract

(57) [Abstract] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

Detailed Description of the Invention [Industrial Application Field] The present invention provides a working hand with an imaging means for recognizing a work object, and an illumination means that operates in synchronization with the imaging by the imaging means, and an image processing means for extracting a region corresponding to the fruits and vegetables to be worked on based on the captured image information and extracting a highlight part that appears brightest within the region, and a detection/I by the image processing means;
The present invention relates to a guiding device for a working machine for fruits and vegetables, which is provided with a guiding means for guiding the working hand toward a direction in which the fruits and vegetables to be worked are located based on a n-information.

[Conventional technology]

The guidance device for this type of fruit and vegetable work machine is configured to guide the work hand in a so-called line-of-sight tracking manner.

By the way, if we take fruits such as mandarin oranges and apples as an example of fruits and vegetables to be worked on, when the work target is illuminated, because its shape is spherical, the closest part of its surface will be larger than its surroundings. It is also known that it becomes bright (appears to glow).

Furthermore, since the amount of reflected light is inversely proportional to the square of the distance, the reflected light from fruits and vegetables located close to the imaging position will be stronger.

Therefore, even when multiple fruits and vegetables grow on top of each other, if the brightest visible part, that is, the highlighted part, is used as a representative position of the fruit and vegetables to be worked on, multiple fruits and vegetables can be grown. Even if the fruits and vegetables overlap each other, the one located on the front side of the plurality of fruits and vegetables will be automatically recognized, and the fruit vegetables located on the front side will be compared to the fruit vegetables located on the rear side. This allows you to approach the situation efficiently while ensuring that it does not become a problem.

Therefore, conventionally, as shown in FIG.
The imaging means is set so as to be able to determine changes in the amount of reflected light from the fruits and vegetables to be worked (as shown in (a) in FIG. For example, as shown in FIG. 8(b), the image 1n image taken in the state of being positioned at 1f (LO) is divided into an area (corresponding only to the fruits and vegetables to be worked on) based on the color of the fruits and vegetables to be worked on, etc. F0), and the eighth
As shown in Figure (C), the brightest (visible highlight part (P1)) within the extracted area (F0) is extracted based on the brightness of the captured image information, and this highlight part (PI) is extracted. is used as information representative of the direction in which the fruits and vegetables to be worked are located.

[Problem that the invention seeks to solve]

However, in the above-mentioned conventional configuration, since the work hand is approached based on image information captured in a state where it is located at its initial position, there are the following disadvantages.

In other words, since images are taken while operating the illumination means in synchronization, as shown in Fig. 1, the amount of reflected light from the fruits and vegetables to be worked on that the imaging means receives is inversely proportional to the square of the distance, so that the amount of light reflected from the fruits and vegetables that are located far away is inversely proportional to the square of the distance. The brighter the area is, the lower the amount of light and the smaller the difference in light amount, so for fruits and vegetables that are located far away, even if the corresponding area is extracted, the difference in highlighted parts will be unclear. Therefore, it has been difficult to accurately determine the positional relationship between the front and rear of fruits and vegetables located far away. On the other hand, the difference in the amount of reflected light from fruits and vegetables located far away is large (for example, in Figure 1, (c)
As shown in , when the amount of light from the illumination means is increased, the amount of reflected light from nearby objects becomes too strong.
There is a risk that the amount of light received by the imaging means will become saturated and a state will arise in which changes in brightness cannot be determined.

The present invention has been made in view of the above circumstances, and its purpose is to accurately recognize the longitudinal positional relationship of a plurality of fruits and vegetables based on the difference in the amount of light received by the imaging means. There is a particular thing.

[Means for solving problems]

The characteristic configuration of the guidance device for a working machine for fruits and vegetables according to the present invention is as follows:
A proximity sensor is provided on the work hand for detecting that the work object has approached within a set distance, and the guiding means is based on image information taken with the work hand positioned at an initial position. an initial approach means for causing the work hand to approach in the extracted direction, and for stopping the work hand in a state in which the work hand is located a set distance in front of the fruits and vegetables to be worked based on the detection information of the proximity sensor; In a stopped state, the direction in which a highlighted portion is located is extracted based on captured image information obtained by setting the amount of illumination of the illumination means so that the amount of light received by the imaging means at the set distance is an appropriate amount of light, and re-imaging the image. It consists of a terminal approach means for approaching the target, and its functions and effects are as follows.

[For production]

In other words, using the proximity sensor installed on the work hand,
By stopping the imaging means and illumination means provided on the work hand at a set distance in front of the fruits and vegetables to be worked on, and re-imaging while illuminating with a brightness according to the set distance, the work hand The positional relationship of a plurality of fruits and vegetables to be worked on, which are located in the distance direction relative to the object, can be detected as an appropriate change in brightness. In other words, if you take an image at a set distance by illuminating with a light amount corresponding to that distance, the brightness change in the area corresponding to fruits and vegetables extracted from the captured image information will change greatly depending on the distance. This will be obtained. Therefore, if you extract the highlighted part based on the large brightness change, even if multiple fruits and vegetables overlap, only the highlighted part of the fruit and vegetables located in the foreground will be extracted, which may cause errors. It is possible to prevent the fruit and vegetables located on the rear side from being approached first.

〔Effect of the invention〕

Therefore, a proximity sensor is provided that detects when the fruit and vegetables to be worked are approached within a set distance, and based on the detection information from this proximity sensor, the fruit and vegetables to be worked are stopped at a set distance in front of the fruit and vegetables to be re-imaged. By doing so, it is possible to position the object on the near side of a plurality of fruits and vegetables that appear to overlap without providing a means for measuring the imaging distance to the fruits and vegetables to be worked on and without using a strong light intensity of the illumination means. I was able to approach things accurately.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in Fig. 4, a boom (1) is attached to the vehicle body (2) so as to be able to rise and fall and turn freely, an auxiliary boom (3) is attached to the boom (1) so as to be able to swing freely in the horizontal direction, and fruit harvesting is carried out. A working cocoon break (4) equipped with a working hand (H) is attached to the tip of the auxiliary boom (3) to constitute a working machine for fruit harvesting. In addition, in the figure,
(5) is a hydraulic cylinder for raising and lowering the boom, (6) is an electric motor for rotating the boom, (7) is an electric motor for swinging the auxiliary boom, and (K) is a box for storing the harvested fruit.

The working manipulator (4) consists of a multi-joint telescopic arm (8) and a working hand (+()) attached to the tip of the telescopic arm (8). ) is the vertical axis center (X) with the electric motor (9a).
It is configured so that it can be rotated around the periphery, swung around a horizontal axis (Y) by an electric motor (9b), and extended and contracted by an electric motor (9c).

To explain the working hand (H), as shown in FIGS. 5 and 6, a vacuum pad (10) made of soft rubber is attached with its suction port facing the front side of the hand.
The vent pipe (11) is fitted externally to the tip of the vent pipe (11).
) is provided with a branch pipe (12) for ventilation, and a telescopic bellows type hose (13) is connected to the branch pipe (12). is bolted to the tip of the telescopic arm (8). However, as shown in FIG. 2, the bellows type hose (13) is connected to an intake pump (14).

Further, there are provided a delivery cutter for cutting off the stem, or so-called stalk, of the fruit captured while being attracted to the vacuum pad (10), and a capture unit case (15).

To explain the trapping part case (15), as shown in FIGS. 5 to 7, the proximal end portion (
15a), a part of the upper cover (15b) fixed to the upper part of the distal end thereof, and a part of the lower cover (15c) fixed to the lower part of the distal end of the base end part (15a), and An air cylinder (16) for sliding operation of the part case (15) is connected to the base end part (15a).
) and the branch pipe (12). Accordingly, the suction port of the vacuum pad (10) is located in the front side of the catcher case (15), and the fruit sucked by the vacuum pad (10) is transferred to the inside of the catcher case (15). It is configured so that it can be freely switched between

The upper cover portion (15b) and the lower cover portion (1
To explain 5c), they are formed into a hemispherical shape so as to cover the outer periphery of the hand, and are large enough to allow one fruit to pass through the area facing the suction port of the vacuum pad (10). An opening (26) for the fruit to pass through is opened. In addition, a slit-like groove (27) through which the fruit stalk is inserted is formed in a portion of the upper cover portion (15b) close to the opening (26), so that when harvesting the fruit, as will be described later. To,
It is designed so that only the fruits to be harvested that have been adsorbed by the vacuum pad (10) can be taken into the trapping part case (15), and the handle can be smoothly moved when taking them (No. 9). (See figure (c)).

Further, an opening for discharging the fruit whose handle has been cut, which is held in the trapping part case (15), is formed on the lower rear side of the lower cover part (15c), and the fruit is placed downward. A shutter (29) for discharging is fixed to the lower part of the proximal end portion (15a) of the trapping part case (15) so as to be openable and closable by an air cylinder (30) for opening and closing.

To explain the delivery deadline cutter, as shown in FIG.
) and the handle support member (18) in the form of a semi-circular strip and a semi-circular arc.
15), and is pivotally connected to the proximal end portion (15a) of the force lid (
17) and the handle support member (18) are respectively projected toward the front side of the hand so as to support the handle of the fruit caught in the trapping part case (15) between them, and the fruit is held between them. It is designed to be able to swing in and out so that the hand can be retracted to the rear side to form a passage for people.

An electric motor (19) for swinging the cutter (17) and handle support member (18) is provided, and an air cylinder (20) for driving the slide of the cutter (17).
The handle is supported between the cutter (17) and the handle support member (18) when the cutter (17) swings toward the projecting side, that is, when the handle is raised, and the supported handle is moved toward the cutter (17). It is designed to be cut by the sliding operation of the (see Fig. 9).

However, the force lid (17) and handle support member (18) are
It is interlocked and connected to the electric motor (19) via a differential mechanism (21), and the cutter (17) and handle support member (
18) If either one of the two contacts the handle first,
While stopping the raising operation of the cutter (17) or handle support member (18) that came into contact with it first, move the handle support member (18) on the other side.
8) or the cutter (17) can continue to operate in an upright position. In other words, the protruding positions of the cutter (17) and the handle support member (18) can be automatically changed in accordance with the position of the handle of the captured fruit.

Further, as will be described later, various sensors are provided to control the operation of each part of the working hand (H).

As shown in FIGS. 5 and 7, a negative pressure sensor (S1) is attached to the branch pipe (12) to detect when fruit is adsorbed to the vacuum pad (11).

As shown in FIGS. 5 to 7, the working hand (H)
A light reflection type proximity sensor (S2) that detects when the fruit approaches within a set distance to the fruit to be harvested, and an image sensor (S3) as an imaging means to detect the direction in which the fruit to be harvested is located. The image sensor (S3) is provided inside the vacuum pad (10) with its detection direction facing the front side of the suction port of the vacuum pad (10), and is also provided at the rear of the upper cover portion (15b). A strobe device (28) is provided that emits light at a set scene in synchronization with the imaging processing performed by ), so that imaging processing can be performed with an appropriate amount of light regardless of the surrounding brightness.

Moreover, as shown in FIGS. 5 and 6, the cutter (17
) and the handle support member (18) are interlocked and connected to the differential mechanism (21), and a potentiometer (S4) for detecting the swinging position of the force lid (17) and the handle support member (18) is interlocked with the differential mechanism (21).
), it is possible to detect the completion of the standing operation and the completion of the retiring operation.

To explain the proximity sensor (S2), a sensor body (22) that houses a light emitting part, a light receiving part, a signal processing part for these, etc. is held at the rear upper part of the branch pipe (12), and the sensor body The optical fiber (23) for emitting and receiving light is fed out from the branch pipe (1).
2) and extends into the inside of the vacuum pad (10) through the inside of the ventilation pipe (11), and a proximity sensor (
The above-mentioned optical fiber (23) as a detection surface of SZ)
The extending end of the image sensor (S3) is supported by the holder (24) for supporting the image sensor (S3) on the upper part of the image sensor (S3) in a state facing the front side of the hand.

Next, a configuration of a control means for automatically causing the harvesting work hand (H) to approach a fruit to be harvested, and for harvesting the fruit by cutting the stalk of the captured fruit will be described.

As shown in FIG. 2, an image processing means (100) for performing image processing based on image information from the image sensor (S3) to determine the positioning direction of the fruit to be harvested, and the working hand (H). a guiding means (101) for guiding the fruit toward the fruit to be harvested, and the trapping part case (15).
A control device (32) is provided constituting each of the harvesting operation control means for controlling the operation of the various parts in order to cut and harvest the stalk of the fruit caught within. In other words, the control device (
In step 32), image information from the image sensor (S3), detection information from the negative pressure sensor (St) and proximity sensor (S2), and detection information from the potentiometer (S4) are input.

Based on these information and previously stored information, the controller (33) for driving the cylinder (5) for raising and lowering the boom, the controller (34) for driving the boom rotation motor (6), A drive controller (35) for the auxiliary boom swing motor (7), each electric motor (9a), (9b) for the telescoping arm (8),
(9c) drive controller (36a).

(36b), (36c), the trapping part case (15
) control valve (
37), an air cylinder (2) for operating the cutter (17);
0), a controller (39) for driving the electric motor (19) for raising and lowering the force lid (17) and the handle support member (18), and the vacuum pad (
10), a control valve (40) for controlling the suction operation, a control valve (41) for the air cylinder (30) for opening and closing the shutter (29), and each of the image sensor (S3) and the strobe device (28). output an operation command to
Detecting the direction in which the fruit to be harvested is located, approaching the work hand (H) toward the fruit to be harvested, harvesting the fruit, and discharging the astringent fruit at a predetermined position. The system is configured to perform each of these automatically.

However, the imaging field of view of the image sensor (S3) is
Movement of the vehicle body (2), rotation and elevation of the boom (1), horizontal rocking of the auxiliary boom (3), and movement of the telescopic arm (
The working hand (H) or the vacuum pad (
The setting is made by changing the direction of 10).

Next, the operation of the control device 2 (32) will be explained based on the flowchart shown in FIG. 3 ((), (1)).

First, the vacuum pad (10) is positioned at a preset initial position and is oriented in the direction of a predetermined range where the fruits to be harvested are located, and then the strobe device (28) is turned on. , the state where the amount of light is maximum (
In FIG. 1, light is emitted (indicated by (a)), and the image sensor (S3) performs imaging processing. Next, the captured image information is taken in, image processing is performed to remove the background such as leaves and leaves, and extract image information that corresponds only to the target fruit.
Find the direction in which the fruit to be harvested is located.

To explain the image processing that removes the background such as fruit and leaves and extracts image information corresponding only to the target fruit, the image information captured by the image sensor (S3) (see Fig. 8 (
b)) is binarized based on the color of the fruit, an area (F0) corresponding only to the fruit (F) is extracted, and the position of the center of gravity (P0) of that area (F0) is determined. , the direction in which the fruit to be harvested is located is the center of gravity (P0)
This is determined based on the coordinate values on the image where is located (see FIG. 8(b)).

Next, since the position of the vacuum pad (10) in the determined direction, that is, the approach start position, and the center of the imaging field of view of the image sensor (S3) match, the coordinate values of the center of gravity (P0) are determined. Based on this, while maintaining the state in which the suction port of the vacuum pad (10) faces in the direction where the center of gravity (P0) is located, that is, toward the fruit to be harvested, the working hand (H) is moved in the direction in which the fruit to be harvested is located. Have them approach while moving at high speed.

However, if a plurality of fruits to be harvested are recognized from the image information captured by the image sensor (S3), the strobe device (28) is made to emit light during image capture processing by the image sensor (S3), so the distance is Since the closer the object is, the brighter it appears, the brighter the object is approached first based on the difference in average brightness of each area (F0) corresponding only to the color of the fruit,
Alternatively, the approach order may be set so that the lower position is approached first. In other words, the closer to the vacuum pad (lO) or the easier to capture the items, the earlier they will be harvested.

Then, after starting the approach, the proximity sensor (S2)
When it is detected that the operator has approached within a set distance from the fruit to be harvested, the operation of the telescoping arm (8) is stopped and the approach of the working hand (H) is temporarily stopped. Move the work hand (H) to the set distance (L1)
(in this example, it is set to about 200 mm), and set the retracting distance so that the amount of light received by the image sensor (S3) does not become saturated and the change in the amount of light with respect to the imaging distance becomes large. and the working distance of the proximity sensor (S2), while reducing the light intensity of the strobe device (28) to a preset light intensity (indicated by (b) in FIG. 1) and re-emitting the light, Perform reimaging processing (Figure 9 (a)
reference).

However, if the proximity sensor (S2) does not operate even if the approach distance exceeds the set value corresponding to the position where the telescoping arm (8) is extended to its maximum extension position,
When it is determined that harvesting is impossible, the telescoping arm (8) is shortened and the working hand (11) is returned to the approach starting position.

Processing until the work hand (H) is stopped at a position a set distance (L1) in front of the work target fruit (F) based on the image information taken at its initial position. , an initial approach means (102) is configured.

Then, by performing image processing again on the image information that has been re-imaging processed, one area corresponding only to the color of the fruit (
F0) is re-extracted, and the re-extracted region (F
0), the highlight part (P
I) (see FIG. 8(c)), correct the approach direction so that the vacuum pad (10) approaches the highlighted part, and then the proximity sensor (S2) Restart the approach while moving at low speed until it is turned on.

With this, move the working hand (H) to the target fruit (F).
) set road for! (1, 1) A final approach means (103) is configured by stopping the vehicle in a position on the near side and approaching the highlight portion (p1) extracted based on image information subjected to re-imaging processing. It is.

However, if there are multiple highlighted parts (p1) in the re-extracted area (F0) (indicated by broken lines in FIG. Select only one highlighted part (p1) to first approach the object located in a preset direction, for example, to the left or below. becomes. In addition, when a plurality of fruits grow adjacent to each other in the front-back direction, the amount of light reflected by the strobe device (28) is greater from the fruit located at the front than from the fruit located at the rear. Therefore, the position information for the fruit located on the rear side is automatically removed, and even if multiple fruits are adjacent, only one highlighted part (p1) is extracted. Working hand (H)
Only the position corresponding to one fruit located on the near side of the image will be automatically extracted.

When the proximity sensor (S2) is turned ON again, the suction operation of the vacuum pad (10) is started, and the working hand (H) is moved forward at a low speed.

After the suction operation of the vacuum pad (10) is started, the working hand (H) is moved forward by a set distance (for example, a distance corresponding to the working distance of the proximity sensor (S2)) toward the fruit to be harvested (9th (See Figure (+7)), it is determined whether or not the fruit has been suctioned based on the detection information of the negative pressure sensor (S1), and if the fruit has been suctioned as desired, the vacuum pad (10) is stopped. , and at the same time, the catching part case (15) is made to protrude forward, the fruit to be harvested is taken into the catching part case (15), and the fruit is positioned within the operating range of the delivery deadline cutting device. (See Figure 9 (c)).

However, if the negative pressure sensor (S1) does not operate, it is determined that harvesting is impossible, and the trapping part case (15) is evacuated to the rear side, and the vacuum band (l)
The suction operation of O) is stopped, the telescoping arm (8) is shortened, and the working hand (H) is returned to the approach starting position.

After taking the fruit into the trapping part case (15),
The motor (
19) in normal rotation. Then, based on the information detected by the potentiometer (S4), the rising operation is temporarily stopped at an intermediate position on the front side of the rising operation completion position, and the suction operation of the vacuum pad (10) is stopped, and the working hand (H) By retracting the entire body a set distance downward toward the front side with respect to the fruit, the cutter (17) is brought into contact with the captured fruit, that is, the force lid is The position of the handle relative to the cutter (17) is corrected so that the length of the handle cut at step (17) is as short as possible (see FIG. 9(d)).

Thereafter, based on the information detected by the potentiometer (S4), the cutter (17) and the handle support member (18) are continuously operated to raise the fruit in the forward direction until the raising operation is completed. The handle is held between the cutter (17) and the handle support member (18), and the handle is cut by the cutter (17) (see FIG. 9(N)).

However, the force lid (17) and the handle support member (18) are
If it cannot be detected that the standing operation has reached that position even after the set time required to reach the completion position or intermediate position has elapsed, it is determined that harvesting is impossible, and the operation is performed based on the detection information by the potentiometer (S4). , the force lid (17) and the handle support member (18) are retracted rearward to their fully retracted positions, the catching part case (15) is retracted to the rear side of the hand, and the vacuum pad (10 ) is returned to the initial position in which the suction port protrudes to the front side of the opening (26) of the trapping part case (15), and then the telescoping arm (8) is shortened to the approach starting position. It will be returned.

After cutting the handle, while maintaining the state in which the fruit is held within the trapping part case (15), the telescopic arm (8)
After returning the working hand (H) to a predetermined position, the harvested fruit is placed in the storage area beside the vehicle body (2) by opening the shaft (29) downward. Discharge into the designated box (K).

After discharging the harvested fruit, the potentiometer (
Based on the detection information by S4), the force lid (17)
and the handle support member (18) is retracted rearward to its retracted completion position, and the catching part case (15)
Retract the hand to the rear side and remove the vacuum pad (
10) is connected to the opening (15) of the trapping part case (15).
26) is returned to the initial position in which it protrudes forward, and the harvesting operation for one fruit is completed.

[Another example]

In carrying out the present invention, the specific configuration of the working hand (H),
The configuration of each part can be changed in various ways, such as the specific configuration of the telescoping arm (8) provided for directing the working hand (H) toward the harvesting target.

Further, in the above embodiment, the case of harvesting fruits was illustrated, but the present invention can also be applied to harvesting of vegetables such as tomatoes. Further, the present invention can be applied to works other than harvesting work, and the present invention is not limited to the above embodiments.

Incidentally, although reference numerals are written in the claims section for convenience of reference to the drawings, the present invention is not limited to the structure of the attached drawings.

[Brief explanation of the drawing]

The drawings show an embodiment of the guidance device for a working machine for fruits and vegetables according to the present invention, in which FIG. 1 is an explanatory diagram of the amount of light of the illumination means, FIG. 2 is a block diagram showing the control configuration, and FIG. 3 ((), (II)
is a flowchart showing the operation of the control device, FIG. 4 is an overall side view of the fruit harvesting machine, FIG. 5 is a cutaway side view of the working hand, FIG. 6 is a cutaway plan view of the same, and FIG. 7 is a front view of the same. figure,
Figures 8 (a), (■), and (71) are explanatory diagrams of image processing;
Figure 9 (a). (0), (C), (D), and (N) are explanatory diagrams of the harvesting operation. (H)...Working hand, (St)...
・Proximity sensor, (S3)... Imaging means, (F0
)......area corresponding to the fruits and vegetables to be worked on, (P1
)...Highlight part, (L1)...
Setting distance, (28)...Illumination means, (100)
...image processing means, (101) ...guidance means, (102) ...initial approach means,
(103)...Terminal approach means.

Claims (1)

[Claims] The working hand (H) is provided with an imaging means (S_3) for recognizing the work object, and an illumination means (28) that operates in synchronization with the imaging by the imaging means (S_3), and a Area corresponding to the fruits and vegetables to be worked on (
an image processing means (100) that extracts a highlight part (P_1) that appears brightest in the area (F_0), and performs the operation based on the detection information by the image processing means (100). A guidance device for a fruit and vegetable working machine, which is provided with a guiding means (101) for guiding a hand (H) toward a direction in which fruits and vegetables to be worked are located, the guidance device being provided with a guiding means (101) for guiding a hand (H) toward the direction in which fruit and vegetables to be worked are located, A proximity sensor (S_2) for detecting proximity is provided on the working hand (H), and the guiding means (101)
While the working hand (H) is positioned at the initial position, the working hand (H) is approached in a direction extracted based on the image information taken, and the working hand (H) is moved based on the detection information of the proximity sensor (S_2). an initial approach means (102) for stopping the H) in a state where it is located a set distance (L_1) in front of the fruits and vegetables to be worked; and in the stopped state, the imaging means (S_3) at the set distance (L_1); The illumination unit (28) sets the illumination amount of the illumination means (28) so that the received light amount is an appropriate light amount, and approaches the direction in which the highlighted portion (P_1) extracted based on the captured image information is re-imaged. A guidance device for a working machine for fruits and vegetables, comprising a terminal approach means (103).