JPS62278917A - Guide apparatus of working machine for fruits - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] 3. Detailed Description of the Invention [Field of Industrial Application] The present invention is directed to fruit vegetables used for growing management and harvesting of fruits such as mandarin oranges and apples, and fruit vegetables such as tomatoes. Regarding a guidance device for a fruit and vegetable work machine for guiding a work hand of a work machine toward a work target, in detail, the color of the work target fruit and vegetables is determined based on image information obtained by capturing the work target fruit and vegetables. The process of detecting the geometric feature points of the region corresponding to A working hand is equipped with a visual device that sequentially executes each of the processes of detecting the ifi area, and a proximity sensor that detects when the fruit and vegetables to be worked are approached within a set distance. The present invention relates to a guiding device for a working machine for fruits and vegetables, comprising a guiding means for guiding toward the geometric feature point until the point is activated, and then guiding toward the highlighted area.

[Conventional technology]

The above-mentioned guidance device for this type of fruit and vegetable work machine is explained using fruits such as tangerines and apples as an example.These fruits have different colors compared to background objects such as fruit tree branches, leaves, or the sky. Therefore, by extracting a specific color area corresponding to the color of the work target from image information depicting the work target,
In addition to being able to separate and identify the work target and other objects,
Since there is a position corresponding to each fruit where light is reflected strongly on the fruit surface, by extracting the 61 highlight areas that appear the brightest within the above specific color area, it is possible to can be used as positional information corresponding to each work object, so it is possible to easily guide the work object to its location without having to measure the distance to the work object.

In order to efficiently perform a predetermined task such as harvesting on each of a plurality of work objects in the same area, the applicant has proposed that the work hand be kept at a predetermined distance from the work object. Detects when the work hand approaches the work target within a set distance so that the work hand can move at high speed until it approaches the work target, and after that, the work hand can reliably capture the work target. The work hand is equipped with a proximity sensor, and until the proximity sensor is activated, the geometric feature point of the area of the specific color, which is the position in front of the work hand where the work object is definitely present, e.g. ,
We have proposed a means for guiding the object toward the highlight area by moving it at high speed toward the center of gravity, etc., and moving it at low speed by the operating distance of the proximity sensor as the proximity sensor operates.

Conventionally, as shown in FIG. 7, the visual device is
When a command to start image processing is received from the guiding device as a guiding means, the image processing is performed, and then a region of a specific color corresponding only to the color of the fruits and vegetables to be worked is extracted, and the geometrical shape of that region is extracted. The calculation of the center of gravity as a scientific feature point and the detection of the highlight area that appears brightest within the area of the specific color extracted above are executed in sequence, and both the calculation of the center of gravity and the detection of the highlight area are performed. When the process is completed, the guidance device is notified of the completion of the process, and the process result is transmitted as guidance information for the working hand. Then, on the guidance device side, after issuing an image processing start command to the visual device, the guidance device enters a resting state (
WAIT), and waits for notification of the end of processing from the visual device, and then begins an approach to guide the working hand toward each of the recognized geometric feature points and the highlight area, which is the subsequent guidance direction. At the same time, for example, they were made to perform predetermined tasks such as harvesting operations.

[Problem that the invention seeks to solve]

However, in the above conventional configuration, the visual device does not start guiding the work hand until the process of determining both the geometric feature points and the highlight area of the area corresponding only to the color of the fruits and vegetables to be worked is completed. Therefore, there was a disadvantage that the waiting time until starting the approach became longer and work efficiency decreased.

By the way, since the process of calculating both the geometric feature points and the 61 highlight areas of the area corresponding only to the color of the fruits and vegetables to be worked on is a process that involves complex arithmetic processing, image processing that can perform high-speed calculations is required. Even if a dedicated numerical calculation processor or the like is used, it is inevitable that it will take some time.

The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to make it possible to efficiently guide a working hand in the direction in which fruits and vegetables to be worked are located.

[Means for Solving the Problems] The characteristic configuration of the guidance device for a working machine for fruits and vegetables according to the present invention is as follows:
The guiding means is configured to start the guiding operation when the visual device completes the process of detecting the geometric feature point, and the guiding means starts the guiding operation of the visual device. The present invention is configured to execute a process of detecting the highlighted area in conjunction with the process, and its functions and effects are as follows.

[For production]

In other words, the working hand is guided in two stages: first, it is guided towards the geometric feature point, and then it is guided towards the highlight area; It is possible to start guiding the working hand if only information regarding the geometrical feature points is available. or,
Even if the work hand is moved at high speed, it takes some time to reach the position of the target fruits and vegetables, so if the highlight area detection processing is performed during this time, the work hand will be directed to the highlight area. The waiting time until the start of guidance can be shortened.

In other words, when the process of detecting the geometric feature points necessary to guide the work hand is completed, the guidance operation to guide the work hand in the direction of the position is started, thereby improving visual perception. In addition to shortening the waiting time until the image processing of the device is completed, while guiding the work hand toward the above-mentioned geometric feature point,
By executing the four-area detection process, the highlight area detection process can be performed simultaneously while guiding the work hand, and the waiting time until the highlight area detection process is completed can be shortened. can.

〔Effect of the invention〕

Therefore, the visual device is made to first start the process of detecting only the above-mentioned geometric feature points necessary to start guiding the working hand, and then performs the process of detecting the highlight area for guiding. , is performed while the working hand is guided toward the geometric feature point, so the conventional
The waiting time of the guiding means can be significantly shortened compared to a case where the guiding operation waits until the visual device successively detects both the geometrical feature point and the highlighted area. As a result, the waiting time until the image processing on the guiding means side is completed is shortened, and work efficiency can be improved.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in Figure 6, a pair of left and right wheels (1), (1
) on the front and rear of the vehicle body (V), a boom (2) having a dogleg shape in side view is attached so as to be able to rise and fall as well as rotate freely, and an auxiliary boom (3) that can freely swing horizontally is attached to the tip of the boom (2). Further, a working manipulator (4) is rotatably attached to the tip of the auxiliary boom (3) to constitute a fruit harvesting working machine as a working machine for fruits and vegetables. In FIG. 6, (5) is a hydraulic cylinder for raising and lowering the boom, (6) is an electric motor for swinging the boom, and (7) is an electric motor for swinging the auxiliary boom.

The manipulator (4) has a multi-jointed telescopic arm (B
) and a work hand (11) for fruit harvesting attached to the tip of the telescoping arm (B). The structure is constructed so that it can be pivoted around the horizontal axis (Y) by an electric motor (9b), and can be expanded and contracted by an electric motor (9c) for expansion and contraction. It is designed to be made.

A color video camera (S0) constituting a visual device (A) for detecting the position of the target fruit (F) is connected to the vertical axis (X) and horizontal axis (Y) of the telescopic arm (B). At the intersection of the working hand (1()
It is stored facing the direction.

To explain the working hand (11), as shown in FIGS. 4 and 5, it has a U-shaped base frame (1
0) is attached to the tip of the telescoping arm (B), and the capture part case (10) is attached to the tip of the base frame (10).
1) is the installation.

Further, a ventilation pipe (12) having a soft rubber vacuum pad (13) fitted externally to the tip thereof is attached to the catching part case (11).
) is slidably fitted and supported in the cylindrical part (lla) of the vent pipe (12), and a branch pipe (14) for ventilation is externally fitted to the base of the vent pipe (12). ) is connected to a telescopic bellows type hose (15). Then, as shown in FIG. 1, the bellows type hose (15)
is connected to the intake pump (16) by piping. Further, a negative pressure sensor (S1) for detecting that the target fruit (F) to be worked on is adsorbed to the vacuum bar 7 (13),
An infrared emitting proximity sensor (S2) is attached to the branch pipe (14) and detects in a non-contact manner that the working hand (H) approaches the target fruit (F) within a set distance. ) is provided with its detection action portion facing forward at the tip of the vacuum pad (13). In FIG. 1, (29) is an electromagnetically operated on-off valve that controls suction and stop of the vacuum pad (13).

To explain the structure of the proximity sensor (S2), as shown in FIGS. 4 and 5, a sensor body (17) housing a light emitting part, a light receiving part, etc. is held within the branch pipe (14). At the same time, an optical fiber (18) that guides light for emitting and receiving light to the sensor main body (17) is connected to the vent pipe (
It extends into the vacuum pad (13) through the interior of the vacuum pad (12), and its extended end is supported at the center of the vacuum pad (13) by a holder (19).

Further, a cutter (20a) for cutting the fruit stalk of the fruit captured while being attracted to the vacuum pad (13), and a cutter (20a) for pressing and supporting the fruit stalk are provided at the distal end of the trapping part case (11). A stalk cutting device (20) consisting of a stalk pressing member (20b) is provided.

To explain the stalk cutting device (20), each of the force lid (20a) and the stalk pressing member (20b),
It is formed into a band-like shape and a semicircular arc shape, and is pivotally connected to the catching part case (11) so as to be rotatable around a coaxial center, and supports the stalk of the caught fruit between them. a state in which the catching part is protruded forward from the catching part case (11);
It is operatively connected to an electric morph (22) via a differential mechanism (21) so that it can be switched to a retracted state to form a path for introducing fruit between them. That is, the force lid (20, 11) and the stalk pressing member (20b
) is in contact with the fruit stem first, it is possible to continue to raise the other one while stopping the raising operation of the contacting one. Further, a limit switch (s3) for detecting the completion of the rising operation of the stem cutting device (20) and a glue mint switch (s4) for detecting the completion of the retiring operation are connected to the input gear (23) of the differential mechanism (21). ) is provided so that it can be operated by a protrusion (24) attached to it. In the figure, (24) is the force cover (20a).
) is an air cylinder for cutting operation, (25) is a protective cover to prevent fruit from being damaged during approach, (
26) is an air cylinder for opening/closing the same, and (27) is an air cylinder for moving the vacuum pad (13) in and out.

Next, a configuration of a means for guiding the working hand (11) toward the fruit (F) to be worked on based on image information captured by the camera (SO) will be described.

As shown in Fig. 1, by image processing the image information from the camera (SO), the brightest area (F0) corresponding only to the color of the work target fruit (F), which will be described in detail later, is shown to be brightest. It functions as a visual device (A) that detects the geometric feature points (p+) of the light area (Pa) and the area (F0) corresponding only to the color of the work target fruit (F), and also serves as a visual device (A) for detecting the image information thereof. and the proximity sensor (S2), negative pressure sensor (S+), limit switch (S
:1), (S1), etc. and various information stored in advance, each air cylinder ( 24), (26), (
27), each electric motor for turning and swinging (6),
(7) A guiding means (for guiding the working hand (1) toward the target fruit (F) to be worked by controlling the operation of each actuator such as (7), (9a), (9b), (9c), etc. B) A control device (28) is provided.

To explain image processing by the visual device (A), as shown in FIGS.
From the color image information (C0) of the fruit to be worked on (F) obtained in step 0) (see Figure 3 (a)), the fruit to be worked on (F) is
) is obtained by extracting the area (F0) corresponding only to the color of the area (FIG. 3 (III)), and the gray level of the color image information (C0) is quantized. The image information tg(C,) (see FIG. 3(C)) is extracted, and this grayscale image information (C2) is masked with the binarized image information (CI) to obtain the work target fruit (F). ) is the brightest highlight in the area (F0) that corresponds only to the color? The center of gravity (PI) as a geometric feature point of the iEf area (P1) and the area (F0) corresponding only to the color of the fruit to be worked on (F) is determined (Figure 3 ( (d) see).

In other words, since the fruit to be worked on (i.e. the fruit to be worked on) and the leaves and leaves surrounding it, or the background sky, etc., have different colors, the color image information tD (C,) from the camera (S0) is used to determine the fruit to be worked in (i.e. Area corresponding only to the color of F) (['.)
By extracting and binarizing, it is possible to extract an area corresponding only to the target fruit (F), and since fruits are generally spherical, the areas where the reflection on the surface is strong are at least as large as each fruit. Since there is one spot for the area (F1), the highlight area (P0) that appears brightest based on the gray level of the grayscale image information (C2) in the area (F1).
If you extract the
This method takes advantage of the fact that the binarized image information (C1) in which the images are overlapped can be used as information representative of the position of each fruit within the recognized area (F0). Furthermore, the center of gravity (P1) of the area (FO) corresponding only to the color of the work target fruit (F) does not necessarily mean that the highlight area (P0) is located only at one location for - fruits. Furthermore, since the location is not necessarily close to the center of the fruit, when the working hand (11) is guided directly toward the highlight area (P0), the proximity sensor (S2) Since there is a risk that the fruit may not operate, first, the fruit is guided toward the center of gravity (P1) in a state where the fruit is definitely present on the front side.

Next, based on the detection information of the highlight area (P0) and the center of gravity (P1),
) to the work target fruit (l?).
The configuration of B) will be explained in detail while explaining the operation of the control device (28).

As shown in FIG. 2, on the visual device (A) side, as the image processing start command is received from the guiding means ([1) side, the camera (S0) is used to display the fruit to be worked on. At the same time as capturing an image of the range in which fruit (F) is located, execution of the above-mentioned image processing is started, and the center of gravity (p1) of the area (F0) corresponding only to the color of the target fruit (F) and the highlight area (P0) are imaged. ) is started. Then, when the process for determining the center of gravity (P1) is completed, the process result is transmitted to the guiding means (B), and thereafter, the process for determining the highlight area (P0) is started. Upon completion of the processing, the guiding means (B) is notified of the processing results, and then the operation of being in a standby state is repeated until the above-mentioned image processing start command is received.

On the other hand, after sending the above-mentioned image processing start command to the visual device (A), the guiding means (B) waits until the visual device (Δ) completes the process of determining the center of gravity (P1). , the work hand (
11) Start processing the approach that induces. As the proximity sensor (S2) operates, it receives and receives information on the highlight area (P0) detected by the visual device (A) while processing this approach. The guiding direction of the working hand (11) is changed to the direction of the highlight area (P0), and as the negative pressure sensor (S1) operates, the stem cutting device (20
) to complete the harvesting operation for one fruit.

Therefore, on the guidance means (B) side, the visual value ff1 (A) only needs to wait until the process for determining the center of gravity (P1) is completed, so the waiting time from the start of image processing until the start of the approach is shortened. The work hand (II
) reaches within a set distance with respect to the center of gravity (P1), the waiting time until the guiding direction is changed to the direction of the by-light area (p0) is also shortened, and the working hand (11) is
It is possible to efficiently guide the work target fruit (F) recognized by the visual device (A).

[Another example]

In the above embodiment, the case where the center of gravity (P1) of the area (F0) corresponding only to the color of the extracted fruits and vegetables is used as the geometric feature point, but for example, the extraction area (F6)
Other geometrical features may be used, such as points equidistant to the boundary of .

Further, in the above embodiment, the structure for harvesting fruits was exemplified, but the present invention can also be applied to other operations such as harvesting vegetables such as tomatoes.

In carrying out the present invention, the specific configuration of the working hand (l()), the specific configuration for guiding the working hand (11), etc.
The configuration of each part can be changed in various ways depending on the work object and work type.

[Brief explanation of the drawing]

The drawings show an embodiment of the guidance device for a working machine for fruits and vegetables according to the present invention, the first drawing is a block diagram showing the configuration of the taste control system, the second drawing is a flow chart showing the operation of the control device, and the third drawing shows an example of the guidance device for a working machine for fruits and vegetables according to the present invention.
Figures (a) to (d) are explanatory diagrams of image processing by the visual device;
FIG. 4 is a cutaway plan view of the working hand, FIG. 5 is a cutaway side view thereof, and FIG. 6 is an overall side view of the fruit collecting machine. FIG. 7 is a flowchart showing the configuration of the conventional means. (F)... Fruits and vegetables to be worked on, (F0)...
...An area corresponding only to the color of the fruits and vegetables to be worked on, (P0)
...Highlight area, (P1) ...Geometric feature point, (S2) ...Proximity sensor, (1
1)...Working hand, (A)...Visual device, (B)...Guiding means.

Claims (1)

[Claims] Based on the image information of the fruits and vegetables (F) to be worked on,
An area (F) corresponding only to the color of the fruits and vegetables to be worked on (F)
A visual device (A) that sequentially executes each of a process of detecting a geometric feature point (P_1) of the area (F_0) and a process of detecting a highlight area (P_0) that appears brightest within the area (F_0). a working hand (II) equipped with a proximity sensor (S_2) for detecting that the work target fruits and vegetables (F) have approached within a set distance;
guiding means (B) that guides the object in the direction of the geometric feature point (P_1) until the proximity sensor (S_2) is activated, and then in the direction of the highlight area (P_0); A guiding device for a working machine for fruits and vegetables, comprising: guiding the guiding means (B) as the visual device (A) completes the process of detecting the geometric feature point (P_1); The visual device (A) is configured to start the operation, and the visual device (A) is configured to perform a process of detecting the highlight area (P_0) as the guiding means (B) starts the guiding operation. Guidance device for working machines for fruits and vegetables.