JPS63119614A - Guide apparatus of working machine for fruits and vegetables - Google Patents

Abstract

(57) [Abstract] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention provides a work hand for grasping a work object.
an imaging means for recognizing a work object, a proximity sensor for detecting that the work object has approached within a set distance;
capture detection means for detecting that the work target has been captured, image processing means for detecting the direction in which the fruits and vegetables are located based on image information captured by the imaging means; a hand control means that is activated accordingly to control the working operation of the working hand; and
guiding the work hand in a direction in which fruits and vegetables detected based on image information taken while the work hand is positioned at an initial position in which the telescoping arm is shortened; and the proximity sensor detects the work hand; The present invention relates to a guiding device for a working machine for fruits and vegetables, which is provided with an arm control means for controlling the operation of the telescoping arm so as to guide the working hand by a set distance even after the operation is performed until the capture detection means detects the detection operation.

[Conventional technology]

The above-mentioned guidance device for this type of working machine for fruits and vegetables is designed to guide the working hand in a so-called line-of-sight tracking manner.

Conventionally, the direction in which the fruits and vegetables are located is detected based on image information taken with the work hand positioned at its initial position, regardless of its distance.

By the way, to explain the image processing means that detects the direction in which fruits and vegetables are located by image processing, for example, based on the color and shape of the fruits and vegetables to be worked on from the captured image information, the area corresponding to the fruits and vegetables is detected. At the same time, the structure is configured to be able to detect the highlight part that appears brightest or the geometric feature point such as the center of gravity within the extracted area, and the geometric feature point indicates the direction in which the fruits and vegetables are located. It will be used as representative location information.

[Problem that the invention seeks to solve]

However, in the above-mentioned conventional configuration, since the work hand is guided based on image information taken while it is located at its initial position, there are the following disadvantages.

In other words, for fruits and vegetables located far from the initial position, the image processing is performed with the resolution of the imaging means reduced, which has the disadvantage of reducing the detection accuracy of the direction in which it is located and increasing the guidance error. .

Therefore, in order to detect with high precision the direction in which fruits and vegetables located far away are located, the imaging means is configured to have high resolution, and the image processing means is configured to have high definition so that its processing pixel density is high. However, in that case, not only would the device configuration be complicated and expensive, but also the time required for image processing would be long due to the handling of high-resolution image information (and, conversely, work efficiency would be reduced). There is a possibility that it will.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to avoid making the device configuration complicated and expensive (in addition, while suppressing a decrease in work efficiency, To enable a work hand to be accurately guided even for fruits and vegetables.

[Means for solving problems]

The characteristic configuration of the guidance device for a working machine for fruits and vegetables according to the present invention is as follows:
The arm control means is configured to stop the guidance if the proximity sensor does not operate while the arm is guided by a set distance from the initial position, and the arm control means is configured to stop the guidance based on image information re-captured in the stopped state. The working hand is configured to guide the working hand in the direction where the fruits and vegetables are located, and its functions and effects are as follows.

[For production]

In other words, fruits and vegetables located at a short distance that are less affected by errors in the direction in which fruits and vegetables are detected based on the captured image information at the initial position are guided as they are based on the detected information at the initial position, and For fruits and vegetables located at a long distance where the influence of errors is large, the direction in which they are located is re-recognized once the work hand has been guided by a set distance.

〔Effect of the invention〕

Therefore, regardless of the distance from the initial position to the fruits and vegetables, the resolving power of the imaging means, the detection accuracy of the image processing means, and the guidance accuracy of the telescoping arm can be adjusted to the position where the work hand is located within the set distance. It can be simplified to the extent that it can be guided up to. As a result, the disadvantage of not being able to accurately guide fruits and vegetables because they are located far away can be overcome without complicating the device configuration and while suppressing a decrease in work efficiency.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in Fig. 3, a boom (1) is attached to the vehicle body (2) so as to be able to rise and fall and turn freely, an auxiliary boom (3) is attached to the boom (1) so as to be able to swing freely in the horizontal direction, and fruit harvesting is carried out. A working manipulator (4) equipped with a working hand (H) is attached to the tip of the auxiliary boom (3) to constitute a working machine for fruit harvesting. In addition, in the figure, (
5) is a hydraulic cylinder for lifting the boom, (6) is an electric motor for boom rotation, (7) is an electric motor for auxiliary boom swinging,
(K) is a box that stores the harvested fruit.

The working manipulator (4) consists of a multi-joint telescopic arm (8) and a working hand (H) attached to the tip of the telescopic arm (8). The telescoping arm (8) is rotated around the vertical axis (X) by the electric motor (9a), and swung around the horizontal axis (Y) by the electric motor (9b). At the same time, the electric motor (9
It is configured to be expanded and contracted in step c).

To explain the working hand (H), as shown in FIGS. 4 and 5, it is equipped with a vacuum pad made of soft rubber (
10) is externally fitted onto the tip of the ventilation pipe (11) with its suction port facing the front side of the hand, and the ventilation pipe (11)
A branch pipe (12) for ventilation is provided at the base of the branch pipe (12), and a telescopic bellows type hose (13) is connected to the branch pipe (12). is bolted to the tip of the telescoping arm (8). However, as shown in FIG. 1, the bellows type hose (13)
It is connected to the intake pump (14) through piping.

Also provided is a delivery cutter for cutting off the stem, or so-called stalk, of the fruit captured while being attracted to the vacuum pad (10), and a capture unit case (15).

To explain the trapping part case (15), as shown in FIGS. 4 to 6, the proximal end portion (
15a), a part of the upper cover (15b) fixed to the upper part of the distal end thereof, and a part of the lower cover (15c) fixed to the lower part of the distal end of the base end part (15a),
Then, an air cylinder (work 6) for sliding operation of the trapping part case (15) is attached to the base end part (15).
a) and the branch pipe (12). Accordingly, the suction port of the vacuum pad (10) is located in the front side of the catcher case (15), and the fruit sucked by the vacuum pad (10) is transferred to the inside of the catcher case (15). It is configured so that it can be switched freely between

The upper cover portion (15b) and the lower cover portion (1
To explain 5c), they are formed into a hemispherical shape so as to cover the outer periphery of the hand, and are large enough to allow one fruit to pass through the area facing the suction port of the vacuum pad (10). An opening (26) is opened for the fruit to pass through. In addition, a slit-like groove (27) through which the fruit stalk is inserted is formed in a portion of the upper cover portion (15b) close to the opening (26), so that when harvesting the fruit, as will be described later. To,
It is designed so that only the fruit to be harvested that has been absorbed by the vacuum pad (10) can be taken into the trapping part case (15), and the handle can be smoothly moved when taking the fruit (No. 8). (See figure (c)).

Further, an opening is formed on the lower rear side of the lower cover part (15c) for discharging the fruit with the handle cut off and held in the trapping part case (15), and the fruit is opened downward. A shutter (29) for discharging is fixed to the lower part of the proximal end portion (15a) of the trapping part case (15) so as to be openable and closable by an air cylinder (30) for opening and closing.

To explain the finger cutting device, as shown in FIG.
) and the handle support member (18) in the form of a semi-circular strip and a semi-circular arc.
The cutter (15) is pivotally connected to the base end portion (15a) of the cutter (15).
17) and the handle support member (18) are respectively projected toward the front side of the hand so as to support the handle of the fruit captured in the trapping part case (15) between them, and the fruit is introduced between them. The hand is swingable in and out so that it can be switched to a state in which the hand is retracted to the rear side to form a passage for the hand.

An electric motor (19) for swinging the cutter (17) and handle support member (18) is provided, and an air cylinder (20) for driving the slide of the cutter (17).
With this, the cutter (17) and the handle support member (18
), and the supported handle is attached to the cutter (17).
) is designed to cut by sliding operation (No. 8).
(see figure).

However, the force lid (17) and handle support member (18) are
It is interlocked and connected to the electric motor (19) via a differential mechanism (21), and the cutter (17) and handle support member (
18) If either one of the two contacts the handle first,
While stopping the raising operation of the cutter (17) or handle support member (18) that came into contact with it first, move the handle support member (18) on the other side.
8) or the cutter (17) can continue to be operated in an upright position. In other words, the protruding positions of the force lid (17) and the handle support member (18) can be automatically changed in accordance with the position of the handle of the captured fruit.

Further, as will be described later, various sensors are provided to control the operation of each part of the working hand (H).

As shown in FIGS. 4 and 5, a negative pressure sensor (S1) that senses that fruit is attracted to the vacuum pad (11) is used as a capture detection means that detects that a work target has been captured. It is attached to the branch pipe (12).

As shown in FIGS. 4 to 6, a light-reflecting proximity sensor (St'') detects when the work hand (H) approaches the fruit to be harvested within a set distance; An image sensor (S1) as an imaging means for detecting the direction in which the fruit is located is provided inside the vacuum pad (10) with its detection direction facing the front side of the suction port of the vacuum band (10). In addition, a strobe device (28) that emits light at a set light amount in synchronization with the insurance processing by the image sensor (S1) is provided at the rear of the upper cover portion (15b), so that an appropriate scene can be seen regardless of the surrounding brightness. It is designed so that image processing can be performed using .

Moreover, as shown in FIGS. 4 and 5, the cutter (17
) and the handle support member (18) are interlocked and connected to the differential mechanism (21), and a potentiometer (S1) for detecting the swinging position of the cutter (17) and the handle support member (18) is interlocked with the differential mechanism (21).
8) The completion of the standing up operation and the completion of the retiring operation can be detected.

To explain the proximity sensor (SZ), a sensor main body (22) housing a light emitting part, a light receiving part, a signal processing part therefor, etc. is held at the rear upper part of the branch pipe (12), and the sensor main body The optical fiber (23) for emitting and receiving light is fed out from the branch pipe (1).
2) and extends into the interior of the vacuum band (10) through the interior of the ventilation pipe (11), and a proximity sensor (
The above-mentioned optical fiber (23) as the detection working surface of S2)
The extending end of the image sensor (S3) is supported by the holder (24) for supporting the image sensor (S3) on the upper part of the image sensor (S1) in a state facing the front side of the hand.

Next, a description will be given of the configuration of a control means for automatically causing the work hand (H) to approach a fruit to be harvested, and for harvesting by cutting the stalk of the captured fruit.

As shown in FIG. 1, an image processing means (100) for processing the image information from the image sensor (S1) to determine the direction in which the fruit to be harvested is located, and the work hand (H) for harvesting. arm control means (10) for controlling the operation of the telescoping arm (8) to guide it toward the target fruit
1), and a control device (32) constituting each of the hand control means (102) that controls the operation of each part in order to cut and harvest the stalk of the fruit caught in the catcher case (15). This is provided. In other words, the control device (32
), the captured image information from the image sensor (S3), the detection information of the negative pressure sensor (S1) and the proximity sensor (S2), and the detection information of the potentiometer (S4) are input.

Based on these information and previously stored information, the controller (33) for driving the cylinder (5) for raising and lowering the boom, the controller (34) for driving the boom rotation motor (6), A drive controller (35) for the auxiliary boom swing motor (7), each electric motor (9a), (9b) for the telescoping arm (8),
(9c) drive controller (36a).

(36b), (36c), the trapping part case (15
) control valve (
37), an air cylinder (2) for operating the cutter (17);
0), a controller (39) for driving the electric motor (19) for raising and lowering the cutter (17) and handle support member (18), and the vacuum pad (
10), a control valve (40) for controlling the suction operation, a control valve (41) for the air cylinder (30) for opening and closing the shutter (29), and each of the image sensor (S1) and the strobe device (28). output an operation command to
Detecting the direction in which the fruit to be harvested is located; causing the work hand (H) to approach the fruit to be harvested; and image information subjected to re-shaping processing when the proximity sensor (S1) does not operate. automatically approach the direction in which fruits and vegetables extracted based on the location are located, that is, the highlighted portion (p1), harvest the fruits, and discharge the harvested fruits at a predetermined position. It is configured so that it can be carried out in a specific manner.

However, the initial position (L) where the telescoping arm (8) is shortened is
0) (see Figure 3).
The imaging field of view in step 3) is based on the movement of the vehicle body (2), the rotation and elevation of the boom (1), the horizontal swing of the auxiliary boom (3), and the movement of the telescoping arm (8) relative to the auxiliary boom (3). Setting is performed by changing the direction of the work hand (H), that is, the vacuum pad (10), by turning or swinging back and forth.

Next, the operation of the control device (32) will be explained based on the flowcharts shown in FIGS. 2(A) and (+7).

First, after placing the vacuum pad (10) in the initial position (L0) and pointing in the direction of a predetermined range where the fruits to be harvested are located, the strobe device (28) is activated. Light is emitted and static image processing is performed by the image sensor (S1). Next, the captured image information is taken in and image processing is performed to remove the background such as grains and leaves and extract image information corresponding only to the target fruit, thereby determining the direction in which the target fruit is located.

To explain the image processing that removes backgrounds such as branches and leaves and extracts image information corresponding only to fruits, image information captured by the image sensor (S3) (Fig. 7 (a)
Reference) is binarized based on the color of the fruit to extract a region (F0) that corresponds only to the fruit (F), and to calculate the position of the highlight part (p1) that appears brightest within that region (F0). By determining the direction in which the fruit to be harvested is located, the direction in which the fruit to be harvested is located is determined based on the coordinate values on the image where the highlighted portion (P1) is located (see FIG. 7 (0)).
.

Next, since the position of the vacuum pad (10) in the obtained direction, that is, the approach start position, and the center of the imaging field of view of the image sensor (S3) match, the coordinates of the highlighted part (P1) are determined. Based on the value, while maintaining the state in which the suction port of the vacuum band (10) faces in the direction where the highlight portion (P1) is located, that is, toward the fruit to be harvested, the work hand (H)
The fruit is approached while moving at high speed in the direction where the fruit to be harvested is located.

However, if a plurality of fruits to be harvested are recognized from the static image information provided by the image sensor (S3), the strobe device (28) is made to emit light during image processing by the image sensor (S1). Since objects that are closer to each other appear brighter, brighter objects are approached first based on the difference in average brightness of each area (F0) corresponding only to the color of the fruit, or The approach order is set so that the lower position is approached first. In other words, the closer to the vacuum pad (10) or the easier to capture the items, the earlier they will be harvested.

Then, after starting the approach, the proximity sensor (SZ)
When it is detected that the target fruit has approached within a set distance, the extension speed of the telescoping arm (8) is switched to a low speed so that the work hand (H) moves at a low speed. After that, the negative pressure sensor (St) detects the fruit (
The proximity sensor (
8 (a) and (b)).

However, the work hand (H) may be replaced by the telescopic arm (8).
The first set distance (L1) corresponds to half of the maximum extension distance of
If the proximity sensor (SZ) does not detect the fruit while guiding the fruit, the operation of the telescoping arm (8) is stopped to temporarily stop the approach of the work hand (11), and in the stopped state, , the strobe device (28
) is re-emitted and re-imaged, the image is processed again, the aforementioned highlight portion (p1) is re-extracted, and the vacuum pad (10) is placed in the re-extracted highlight portion (p1). The approach direction is corrected and the approach is restarted so that the vehicle approaches the target (see FIG. 7 (c)).

Note that if there are multiple highlighted portions (p1) in the re-extracted area (F0) (in FIG. 7(c),
(shown with a broken line) determines that multiple fruits are growing adjacent to each other in a plane, and approaches those located in a preset direction, such as to the left or below, first. Thus, only one highlighted portion (P1) is selected. However, if multiple fruits grow adjacent to each other in the front-back direction, the amount of light reflected by the strobe device (28) will be greater from the fruit located at the front than from the fruit located at the rear. Therefore, the position information for the fruit located on the rear side is automatically removed, and even if multiple fruits are adjacent, only one highlighted part (P1) is extracted. Only the position corresponding to one fruit located on the near side with respect to the work hand (H) will be automatically extracted.

As the proximity sensor (SZ) is turned ON again, the suction operation of the vacuum pad (10) is started as described above, and the movement speed of the work hand (H) is switched to low speed, and then the work hand (H) moves forward. It will be necessary to do so.

After the suction operation of the vacuum pad (10) is started, the work hand (1) is moved forward by a set distance (for example, a distance corresponding to the working distance of the proximity sensor (SZ)) toward the fruit to be harvested (the eighth (See Figure (II1)), it is determined whether or not the fruit has been sucked based on the detection information of the negative pressure sensor (S1).

However, even if the approach distance exceeds the second set distance corresponding to the position where the telescoping arm (8) is extended to its maximum extension position, if the proximity sensor (SZ) does not operate, harvesting is not possible. If it is determined that there is, the telescopic arm (8) is shortened and the work hand (H) is returned to the approach starting position.

Then, the work hand (H) is guided based on image information taken at its initial position or at a position advanced to a set distance (L1), and the vacuum pad (10) adsorbs the fruit (F). The arm control means (101) is constituted by the processing up to the detection operation of the negative pressure sensor (St).

The negative pressure sensor (S1) is turned on and the fruit (
F), use the vacuum pad (1)
0), and then the catching part case (15)
protrudes forward and captures the fruit to be harvested into the trapping part case (15) so that the fruit is located within the operating range of the delivery cutoff device (see Fig. 8).
(See C).

However, if the negative pressure sensor (St) does not operate, it is determined that harvesting is impossible, and the trapping part case (15) is evacuated to the rear side, and the vacuum band (1
0), the telescopic arm (8) is shortened, and the working hand (l()) is returned to the approach starting position.

After taking the fruit into the trapping part case (15),
The motor (
19) in normal rotation. Then, based on the information detected by the potentiometer (S1), the raising operation is temporarily stopped at an intermediate position before the completed position of the raising operation, and the suction operation of the vacuum pad (10) is stopped, and the work hand ( H) By retracting the entire body a set distance downward toward the front of the fruit, the cutter (17) is brought into contact with the captured fruit, that is, the force lid ( The position of the handle relative to the cutter (17) is corrected so that the length of the handle cut at step 17) is as short as possible (see FIG. 8 (=)).

Thereafter, based on the information detected by the potentiometer (S4), the force lid (17) and the handle support member (18) are continuously operated to raise the fruit towards the front until the raising operation is completed. The handle is held between the force lid (17) and the handle support member (18), and the handle is cut with the cutter (17) (see Figure 8 (ladle)).
.

However, the force lid (17) and the handle support member (18) are
If it cannot be detected that the standing operation has reached that position even after the set time required to reach the completion position or intermediate position has elapsed, it is determined that harvesting is impossible, and the operation is performed based on the detection information by the potentiometer (S4). , the cutter (17) and the handle support member (18) are retracted rearward to their retiring completion positions, the catcher case (15) is retired rearward of the hand, and the vacuum pad (10) is retracted. After returning to the initial position in which the suction port projects to the front side of the opening (26) of the catching part case (15), the telescoping arm (8) is shortened and returned to the approach starting position. .

After cutting the handle, while maintaining the state in which the fruit is held within the trapping part case (15), the telescopic arm (8)
After returning the working hand (H) to the predetermined position, the shutter (29) is opened downward, and the harvested fruits are placed on the side of the vehicle body (2). Discharge into box (K).

After discharging the harvested fruit, the potentiometer (
Based on the detection information by S4), the cutter (17)
and the handle support member (18) is retracted rearward to its retracted completion position, and the catching part case (15)
Retract the hand to the rear side and remove the vacuum pad (
10) is connected to the opening (15) of the trapping part case (15).
26) is returned to the initial position in which it protrudes forward, and the harvesting operation for one fruit is completed.

Thus, after the negative pressure sensor (S1) is turned ON, the hand control means (102) is configured to perform a process of controlling the operation of each part of the working hand (H) to perform a harvesting operation.

After the harvesting operation for one fruit is completed, return the working hand (H) to the initial position (L0), and
For example, by determining whether or not there is a fruit to be harvested based on the information on the fruit (F) extracted based on the image information taken at the initial position i (L0), it is possible to determine whether the work has been completed or not. Determine whether If there is a fruit to be harvested, the above-described series of processes is repeated to perform the harvesting operation on the next fruit (F).

[Another example]

In the above embodiment, the set distance (L1) at which the operation of the telescoping arm (8) is stopped and reimaging processing is performed when the proximity sensor (S2) does not perform the detection operation is set to the telescoping arm (8).
8) is set to half of the maximum extension distance, but the specific numerical value of the set distance (shi1) can be changed in various ways. For example, the set distance (L1) may be set to
), and if the proximity sensor (S2) does not operate until the set distance is reached, the direction in which the fruits and vegetables are located is re-detected each time. The guiding direction may also be corrected. Furthermore, the set distance (L1) does not have to be a distance that equally divides the maximum extension distance of the telescoping arm (8).

In carrying out the present invention, the configuration of each part can be changed in various ways, such as the specific configuration of the working hand (H) and the specific configuration of the telescoping arm (8) provided for guiding the working hand (H) toward the harvesting target. .

Further, in the above embodiment, the case of harvesting fruits was illustrated, but the present invention can also be applied to the work of harvesting vegetables such as tomatoes. Further, the present invention can be applied to works other than harvesting work, and the present invention is not limited to the above embodiments.

Note that although reference numerals are written in the claims section for convenience of reference to the drawings, the present invention is not limited to the structure shown in the accompanying drawings.

[Brief explanation of the drawing]

The drawings show an embodiment of the guidance device for a working machine for fruits and vegetables according to the present invention, and FIG. 1 is a block diagram showing the control configuration, and FIG.
<), (11) is a flowchart showing the operation of the control device, Figure 3 is an overall side view of the fruit harvesting work machine, Figure 4 is a cutaway side view of the working hand, Figure 5 is a cutaway plan view of the same, Figure Figure 6 is a front view of the same, Figure 7 ((), (TI), (
C) is an explanatory diagram of image processing, Figure 8 (A), (II)
, (, C), (=), (N) are explanatory diagrams of the harvesting operation. (H)...Working hand, (S1)...
Capture detection means, (S2)... Proximity sensor, (S
1)... Imaging means, (L0)... Initial position, (L1)... Set distance, (8)...
... Telescopic arm, (100) ... Image processing means, (101) ... Arm control means, (102)
・Old...Hand control means.

Claims (1)

[Claims] A work hand (H) for capturing the work object is provided at the tip of the telescoping arm (8), and an imaging means (S_3) for recognizing the work object is attached to the work hand (H). A proximity sensor (S_2) for detecting that the work target has approached within a set distance, and a capture detection means (S_1) for detecting that the work target has been captured, respectively, are provided,
Image processing means (100
), a hand control means (102) that is activated when the capture detection means (S_1) performs the detection operation and controls the work operation of the work hand (H);
H) to the initial position (L) where the telescopic arm (8) is shortened.
The work hand (H) is guided in the direction in which the fruits and vegetables detected based on the image information taken in the state of being positioned at the proximity sensor (S_2) are guided.
) arm control means (101) for controlling the operation of the telescoping arm (8) in order to guide the work hand (H) by a set distance until the capture detection means (S_1) operates for detection. A guiding device for a working machine for fruits and vegetables, wherein the proximity sensor (S_2) is activated for detection while guiding the arm control means (101) by a set distance (L_1) from the initial position (L_0). If not, the guide is configured to stop, and the work hand (H) is configured to guide the work hand (H) in the direction in which the detected fruits and vegetables are located based on the image information re-imaged in the stopped state. A guidance device for a working machine for fruits and vegetables.