JPH0582163B2 - - Google Patents

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention provides an imaging means for taking an image of fruits and vegetables, extracting the fruits and vegetables based on image information captured by the imaging means, and determining the direction in which the fruits and vegetables are located. An image processing means for obtaining a desired image, and an illumination means for illuminating in the direction of the imaging field of view when the image is taken by the imaging means, and a working hand attached to a telescoping arm is directed in the direction in which the extracted fruits and vegetables are located. The present invention relates to a guidance device for a working machine for fruits and vegetables, which is provided with arm control means for operating the telescoping arm based on information from the image processing means for guidance.

[Conventional technology]

To explain the image processing in the guidance device of this type of fruit and vegetable working machine mentioned above, for example, by binarizing the captured image information based on the color of the fruit and vegetables to be worked, it is possible to handle only the fruits and vegetables to be worked. Then, by performing processing such as determining the geometric feature points such as the center of gravity of the extracted area, fruits and vegetables are extracted, and the direction in which they are located is determined.
A method of determining the coordinate value based on the relationship between the coordinate value in the image information and the working machine will be used.

In other words, it is necessary to perform image processing with the brightness of the captured image information normalized, but since the guidance device of this type of fruit and vegetable work machine is often used outdoors, it is difficult to use natural light. The issue is the impact. In other words, since the brightness of natural light is unstable, the color temperature may vary, and depending on the position of fruits and vegetables, the imaging conditions may become so-called backlit conditions.

Therefore, in order to eliminate the influence of natural light, an illumination means is provided that illuminates the field of view of the imaging means when the imaging means takes an image.

Conventionally, in order to sufficiently eliminate the influence of natural light, the illumination means used has a large amount of illumination.

[Problems that the invention attempts to solve]

However, the conventional configuration described above has the following disadvantages, and there is room for improvement.

In other words, the distance range in which the telescoping arm to which the work hand is attached is extendable is limited, and for example, if the amount of light from the illumination means is increased too much, fruits and vegetables located further away than the working distance range of the telescoping arm, that is, the work hand, may be You will also recognize that. the result,
Fruits and vegetables that are located outside the reachable range of the work hand are also guided, resulting in unnecessary guidance and a disadvantage that work efficiency is reduced.

The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to improve the brightness of captured image information so that an image processing means can extract fruits and vegetables located within a range where a work hand can be guided. By optimizing this, it is possible to prevent the work hand from being guided to fruits and vegetables located outside the range in which the work hand can be guided.

[Means for solving problems]

A characteristic configuration of the guidance device for a working machine for fruits and vegetables according to the present invention is such that the illumination means illuminates the fruits and vegetables extracted by the image processing means with a source light amount within a range that can guide the work hand. Its functions and effects are as follows.

[Effect]

That is, by setting the light intensity of the illumination means so that the fruits and vegetables extracted by the image processing means are within a range where the work hand can be guided, the fruits and vegetables extracted by the image processing means are located at a distance where the work hand cannot be guided. It is possible to prevent the work hand from being guided even to fruits and vegetables.

〔Effect of the invention〕

Therefore, by configuring the illumination means to illuminate with a light intensity such that the distance range of the fruits and vegetables extracted by the image processing means is within the guidance distance range of the work hand, it is possible to illuminate the fruits and vegetables in the range where the work hand cannot be guided. This has made it possible to prevent waste caused by the work hand being guided. Furthermore, by illuminating the work hand with an amount of light that corresponds to the guiding range of the work hand, the amount of light of the illumination means can be optimized, so that it is possible to prevent the illumination means from becoming unnecessarily large.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in Fig. 5, a boom 2, which has a dogleg shape in side view, is attached to a vehicle body V equipped with a pair of left and right wheels 1, 1 at the front and rear so that it can move up and down as well as turn. Flexible auxiliary boom 3
A working manipulator 4 is attached to the tip of the auxiliary boom 3 so as to be able to swing up and down and rotate, thereby forming a working machine for harvesting fruits such as mandarin oranges. In the figure, 5 is a hydraulic cylinder for raising and lowering the boom, 6 is an electric motor for swinging the boom, and 7 is an electric motor for swinging the auxiliary boom.

The manipulator 4 consists of a multi-joint telescopic arm 8 and a fruit harvesting work hand H attached to its tip, and the telescopic arm 8 is rotated around the vertical axis X by an electric motor 9a. Moreover, it is constructed so that it can be swung around a horizontal axis Y by an electric motor 9b, and it can be expanded and contracted by an electric motor 9c for expansion and contraction.

Further, as shown in FIGS. 5 and 6, a color video camera S ₀ is used as an imaging means for imaging fruits and vegetables.
and a strobe light emitting device A serving as an illumination means for illuminating the viewing direction of the color video camera _S0 with a set light amount, such that the viewing direction thereof faces the direction of the work hand H. It is stored at the intersection of the vertical axis X and the horizontal axis Y of the telescopic arm 8.

However, as shown in FIG. 1, the amount of light emitted by the strobe light emitting device A is different from that of the work hand H when the telescoping arm 8 is shortened to its initial position.
When the fruit is located where the tip of the color video camera S0 is located, the amount of light received by the color video camera _S0 is maximum, and the work hand H is in a state where the telescoping arm 8 is extended to a substantially linear state.
The amount of light received is set to be the lower limit at which the image processing means 100 (described later) can extract the fruit when the fruit is located where the tip of the fruit is located. That is, the distance range between the initial position L ₀ and the maximum extension distance L ₁ of the work hand H is a distance range in which the work hand H can be guided, and the image processing means 10
0 is configured to extract fruits located within that distance range.

To explain the work hand H, as shown in FIGS. 7 and 8, the base frame 10, which is in the shape of U when viewed from above, is attached to the tip of the telescoping arm 8, and the base frame 10 is A trap case 11 is attached to the tip.

In addition, a ventilation pipe 12 having a vacuum pad 13 made of soft rubber externally fitted on the tip thereof is supported by being slidably inserted into the cylindrical part of the trapping part case 11, and a base part of the ventilation pipe 12 is supported. , a branch pipe 14 for ventilation is fitted onto the outside, and the branch pipe 14 is fitted with a
A telescopic bellows type hose 15 is connected. As shown in FIG. 2, the bellows type hose 15 is connected to an intake pump 16.

A negative pressure sensor _S1 is attached to the branch pipe 14 to detect that the fruit F has been adsorbed to the vacuum pad 13, and also to detect that the work hand H has approached the fruit F within a set distance. An infrared light projecting type proximity sensor S ₂ for detecting contact is provided with its detection action portion facing forward of the tip of the vacuum pad 13 .

To explain the structure of the proximity sensor _S2 , the sensor body 1 houses a light emitting part, a light receiving part, etc.
7 is held in the branch pipe 14, and an optical fiber 18 that guides light for emitting and receiving light to the sensor main body 17 is extended into the vacuum pad 13 through the inside of the ventilation pipe 12, and its extending end is is supported by a holder 19 at the center of the opening of the vacuum pad 13.

In addition, the trapping part case 11 includes a clipper-shaped cutter 20a formed in a semi-circular strip shape and a semicircular arc shape for cutting the peduncle, so-called stalk, of the fruit captured while being attracted to the vacuum pad 13. A handle cutting device 20 is provided in which a handle supporting member 20b formed in a semi-circular band shape and a semi-circular arc shape, which presses and supports the handle, is pivotably pivoted around a coaxial core.

To explain the stalk cutting device 20, there are two states in which the cutter 20a and the stalk support member 20b are projected forward from the trapping part case 11 to support the stalk of the captured fruit therebetween, and a state in which the cutter 20a and the stalk support member 20b are projected forward from the trapping part case 11 to support the stalk of the captured fruit between them. It is interlocked and connected to an electric motor 22 via a differential mechanism 21 so that it can be switched to a state in which it is retracted to the rear side to form a passage for introducing the fruit. In other words, when either the cutter 20a or the handle support member 20b comes into contact with the handle first, it is possible to stop the raising operation of the contacting one and then continue to raise the other one. There is. Further, the limit switch S ₃ for detecting the completion of the raising operation of the handle cutting device 20 and the limit switch S ₄ for detecting the completion of the retracting operation are each operated by a protrusion 24 attached to the input gear 23 of the differential mechanism 21. It is designed to be operated. In the figure, 24 is an air cylinder for cutting operation of the cutter 20a, 25 is a protective cover for preventing damage to the fruit F while guiding the work hand H,
26 is an air cylinder for opening and closing the same, and 27 is an air cylinder for operating the vacuum pad 13 to move in and out.

Next, a description will be given of the configuration of the image processing means 100 that extracts the fruit F and determines the direction in which it is located based on the image information captured by the camera _S0 .

As shown in FIG. 2, a subtracter obtains an image signal for only the color of fruit F by subtracting the red component color signal R and the blue component color signal B output from the camera _S0 . A ₁ , the image signal R-B output from the subtracter A ₁ is calculated based on the brightness threshold (Vref) corresponding to the maximum extension distance L ₁ of the work hand H, as shown in FIG. A comparator A 2 that obtains binarized image information C ₁ (see FIG. 4 B) that extracts an area F ₀ corresponding only to the fruit F by binarizing it, and the brightness output from _the camera S ₀ . A/D converter A ₃ , which converts the luminance signal Y as information (see FIG. 4 A) into grayscale image information C ₂ , which is quantized based on its gray level (see FIG. 4 C); a memory M for storing digitized image information C ₁ and the grayscale image information C ₁ ; and
Each image processing control unit (CPU) is provided for determining the direction in which the area _F0 is located, that is, the direction in which the fruit F is located, based on the image information stored in the memory M, and controlling the operation of each part. be.

In other words, since the color of the fruit F, the branches and blades around it, or the background sky, etc. is different from that of the fruit F, the image information captured by the camera S ₀
By extracting the region F ₀ corresponding only to the color of the fruit from C ₀ (see Figure 4 A), only the fruit F can be extracted. Moreover, fruits are generally spherical, and there is at least one location on each fruit where the reflection on the surface is strong when illuminated by the strobe device A. Therefore,
By extracting the highlight portion P ₁ (see FIG. 4 D) that appears brightest based on the gray level of the grayscale image information C ₁ in the extracted region F ₀ , the information on that position can be used as the extracted fruit. By utilizing the fact that it can be used as information indicating the direction in which each of F is located, the control unit (CPU)
The direction in which the highlighted part P ₁ is located is the fruit F
Processing is performed to find the direction in which . However, the strobe light emitting device A is controlled to emit light in synchronization with the imaging operation of the camera S ₀ based on a command from the control unit (CPU).

In the figure, reference numeral 28 indicates that the work hand H is connected to the fruit F based on the detection information of the image processing means 100.
29 is a solenoid valve for suction operation of the vacuum pad 13.

That is, the control device 28 includes the image processing means 100, the negative pressure sensor S ₁ , the proximity sensor
Each air cylinder ₂ operates the lifting cylinder 6, the vacuum pad 13, and the work hand H based on the detection information of S 2 and the limit switches S ₃ and S ₄ and various information stored in advance.
4, 25, 26, solenoid valve 29, electric motors 6, 7, 9a, 9b, 9c for turning and rocking, etc., so that the work hand H can move the extracted fruit F. The telescopic arm 8 is telescopically operated so as to be guided in the direction in which the fruit F is located, and the operation of each part is controlled in order to harvest the fruit F.

Hereinafter, the operation of the control device 28 will be explained based on the flowchart shown in FIG.

First, with the telescopic arm 8 positioned at the approach start position in the most shortened state, the camera is illuminated with the strobe light emitting device A.
Image processing is performed at _S0 , and the direction in which the fruit F is located is determined by the image processing means 100.

Next, the vacuum pad 13 is moved to the catching part case 11 so that the working hand H is linearly projected in the direction extracted by the image processing means 100, that is, in the direction in which the highlighted portion _P1 is located. In a state where it protrudes further forward than the
The approach is started by extending the telescoping arm 8 until the proximity sensor _S1 is activated.

However, if the proximity sensor S ₁ does not operate even after extending the telescoping arm 8 to its maximum extension distance, which is the maximum guidance distance L ₁ (see Figure 1) of the work hand H, it is determined that harvesting is not possible. Then, the telescopic arm 8 is shortened and returned to the original approach starting position.

As the proximity sensor S ₁ operates, the vacuum pad 13 starts suction operation, and the telescoping arm 8 is moved to the proximity sensor until the negative pressure sensor S ₁ operates and detects fruit suction. Extend the installation distance set based on the working distance of S ₁ toward the front. However, if the negative pressure sensor _S1 does not operate even after extending the set distance, it is determined that the capture has failed, and the suction operation of the vacuum pad 13 is stopped, and the telescoping arm 8 is shortened to return to its original position. Return to the approach starting position.

As the negative pressure sensor _S1 operates, the extension of the telescoping arm 8 is temporarily stopped, and then the vacuum pad is moved so that the fruit F is attracted to the vacuum pad 13 and taken into the operating range of the stalk cutting device 20. 13 to the telescopic arm 8 side, the telescopic arm 8 is further moved forward at low speed and stopped.

Then, based on the information detected by the limit switch _S3 , the cutter 20a and the handle support member 20b are erected toward the front side of the hand, and the handle is supported between them.

However, if the limit switch _S3 does not operate, it is determined that harvesting is impossible, and after the handle cutting device 20 is laid down on the side of the telescopic arm 8, the vacuum pad 13 is made to protrude to the front side of the trapping part case 11, The suction operation is stopped and the telescopic arm 8 is returned to the approach starting position.

When the standing operation is completed, the suction operation of the vacuum pad 13 is stopped and the telescopic arm 8 is retreated a set distance downward in front of the fruit F, thereby shortening the stalk of the fruit F. After correcting the position of the handle cutting device 20 with respect to the handle, the cutter 20a is operated to cut the handle.

After cutting the handle, the vacuum pad 13
The suction operation is restarted, and while maintaining the state in which the fruit F whose stalk has been cut is adsorbed and held by the vacuum pad 13, the stalk cutting device 20 is laid down, and the telescopic arm 8 is shortened to return to its original approach. Return the work hand H to the starting position.

After returning the work hand H to the approach start position, the vacuum pad 13 is made to protrude further forward than the catching case 11, thereby ejecting the harvested fruit F and completing the harvesting operation for one fruit F. do.

After finishing the harvesting operation for one fruit F, for example, at the approach start position,
Based on the number of extracted fruits F, their positions, etc., it is determined whether or not there are any fruits to be harvested, thereby determining whether or not to end the work, and if there are fruits to be harvested, the above-mentioned The process will be repeated.

[Another example]

In the above embodiment, the image processing means 100 extracts the area based on the color of the fruit F, and the direction in which the fruit F is located is extracted based on the color of the fruit F.
Highlight part that appears brightest within F ₀
Although we have exemplified a case in which the direction in which P ₁ is located is determined, for example, the shape of the fruit F may be extracted as a feature, or the geometric feature point such as the center of gravity of the extracted area F ₀ may be determined as the direction in which it is located. The specific configuration of the image processing means 100 may be modified in various ways.

Further, in the above embodiment, the color video camera S ₀ as an imaging means and the strobe light emitting device A as an illumination means are provided on the proximal end side of the telescoping arm 8, but for example, the work hand H The specific configuration can be changed in various ways, such as providing a

Further, in the above embodiment, the telescoping arm 8 is configured as a multi-jointed arm, but the specific configuration can be modified in various ways.

Further, in the above embodiment, the present invention is applied to a work machine for harvesting fruits, but it can also be applied to various works such as harvesting vegetables such as tomatoes. Specific configuration, working hand H
The configuration of each part, such as the specific configuration for guiding, can be changed in various ways depending on the work object and work type.

Incidentally, although reference numerals are written in the claims section for convenient comparison with the drawings, the present invention is not limited to the structure shown in the accompanying drawings by the reference numerals.

[Brief explanation of drawings]

The drawings show an embodiment of the guiding device for a working machine for fruits and vegetables according to the present invention, FIG. 1 is a drawing showing the relationship between the light amount of the illumination means and the guiding distance of the working hand, and FIG. 2 shows the image processing means and the arm. A block diagram showing the configuration of the control means, FIG. 3 is a flow chart showing the operation of the control device, FIGS. 4A to 4D are explanatory diagrams of image processing,
Fig. 5 is an overall side view of the working machine for fruits and vegetables, Fig. 6 is a schematic front view of the telescoping arm showing how the imaging means and illumination means are attached, Fig. 7 is a cutaway plan view of the working hand, and Fig. 8 is a cutaway side view of the same. _S0 ...Imaging means, A...Illuminating means, H...Working hand, 8...Extendable arm, 100...Image processing means, 101...Arm control means.

Claims (1)

[Claims] 1 Imaging means S ₀ for imaging fruits and vegetables, the imaging means
An image processing means 100 that extracts fruits and vegetables based on the captured image information of S ₀ and determines the direction in which the fruits and vegetables are located, and an illumination means A that illuminates in the direction of the imaging field of view when the imaging means S ₀ takes an image. The telescoping arm 8 is operated based on information from the image processing means 100 in order to guide the work hand H attached to the telescoping arm 8 toward the direction in which the extracted fruits and vegetables are located. This is a guiding device for a working machine for fruits and vegetables, which is provided with an arm control means 101, in which the illumination means A is arranged so that the fruits and vegetables extracted by the image processing means 100 are within a range in which the working hand H can be guided. A guidance device for a fruit and vegetable work machine configured to illuminate with a desired amount of light.