JPH1051668A - Image pickup device for agricultural robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify a work object by placing either of two kinds of interference filters whose image density is changed before a CCD camera in a momentarily replaced way and entering an image continuously. SOLUTION: In the case of image pickup by a visual sensor 1, a filter motor 45 drives interference filters 3a, 3b and two kinds of the interference filters 3a, 3b whose wavelength bands are respectively 550nm and 850nm pass alternately a front face of a CCD camera 2. A filter position sensor 46 senses the passing position and the camera picks up an image through the two kinds of the interference filters 3a, 3b replaced momentarily so as to enter two kinds of density images continuously. In the image processing by the received image, a density ratio by the reflectance by two wavelength bands is obtained and a computed image is obtained based on computed values resulting from multilication of a luminance level by the ratio and a vegetable such as cucumber is properly recognized by using a binary image obtained by binarizing the arithmetic image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus for an agricultural robot, and more particularly to an image pickup apparatus for an agricultural robot. Belongs to technical fields such as those for identifying

[0002]

2. Description of the Related Art In an agricultural robot provided with a CCD camera as a two-wavelength type visual sensor capable of sliding and moving in an up-and-down direction, a near-visible light range is required. By inputting an image of a work object through two types of interference filters having different spectral reflection characteristics arranged in front of a CCD camera having a sensitivity up to an infrared region, two types of light transmitted through the two types of interference filters are transmitted. A density value image is calculated, and the calculated image is binarized to identify a work object.

[0003] However, conventionally, when imaging is performed by using these two types of interference filters, there is a time interval between two input images that are obtained by exchanging the two types of interference filters. Optical problems associated with change, 2
There are physical problems associated with the replacement of the different types of interference filters, and it has been extremely difficult to adjust these problems and input an image under the same conditions.

Therefore, according to the present invention, two types of interference filters are disposed in front of a CCD camera so that they can be replaced instantaneously.

[0005]

According to the present invention, an image density value changes due to transmission on a front surface of a CCD camera 2 as a two-wavelength type visual sensor 1 for recognizing a work object.
The types of interference filters 3a, 3b are arranged so that they can be replaced instantaneously, and each of the replaced interference filters 3a, 3b is replaced.
The configuration of an imaging device of an agricultural robot that continuously inputs images via 3b.

[0006]

According to the above configuration, in the visual sensor 1 of the agricultural robot, for example, a front surface of the CCD camera 2 having a sensitivity from a visible light region to a near-infrared region for imaging a work target is provided. By combining two types of interference filters 3a and 3b having different spectral reflection characteristics with different wavelengths and rotating the same by a motor or the like, the two types of interference filters 3a and 3b that are instantaneously replaced are located at the positions where they match on the front surface of the CCD camera 2. It is detected by a filter position sensor or the like, and at the time of this detection, two types of density value images can be successively input through the respective interference filters 3a and 3b.

As described above, when two types of interference filters 3a and 3b having different spectral reflection characteristics are instantaneously replaced on the front surface of the CCD camera 2, this position is detected and transmitted through each of the interference filters 3a and 3b. Since two types of density value images can be continuously input, there is an optical difference between the two input images due to a change in sunlight or the like, or replacement of the interference filters 3a and 3b. There is almost no accompanying physical difference, and it is possible to input an image in a state very close to the same condition, and it is possible to identify a work target by binarizing a calculation image obtained by calculating this density value image. .

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings, referring to an agricultural robot for harvesting fruits and vegetables as work objects. A pair of left and right front wheels 10a and rear wheels 10b are provided at lower front and rear positions of the chassis 9, respectively.
And a wheel drive motor 11 for each of the four wheels 10b and the rear wheel 10b, and also supports the steering wheel so as to be steerable and revolvable with respect to the traveling direction, and controls the front and rear wheels 10a and 10b separately. A traveling device 13 is provided by providing a steering angle sensor 12 for detecting a steering angle, and storing a battery 13a as a power source between the front and rear wheels 10a and 10b.

A front cover 14 containing a controller and the like for controlling the operation of the agricultural robot is provided at a front position on the undercarriage 9, and an image of cucumber as fruit and vegetables is taken behind the front cover 14. The work revolving drum 15 is mounted on a visual sensor 1 by a CCD camera 2 and a manipulator 5 for grasping and removing cucumber by the image of the visual sensor 1. A rear cover 16 is provided at a rear position on the undercarriage 9 on the rear side of the work turning cylinder 15 and the rear cover 16 stores and discharges a carry for storing harvested cucumbers. The work device 17 is configured by disposing the carry control unit 16a. After this, cover 1
6, an operation panel 17a of the robot is provided on the rear side surface.

Since the manipulator 5 performs an action similar to the movement of a human arm and a hand by multiple joints, each of the manipulators 5 includes a shoulder joint S, an arm joint A, a wrist joint W, and the like from the base side to the distal end side. The joints are connected and arranged. The shoulder joint S is provided with a shoulder motor 18 by a servo motor in a shoulder joint for rotating the manipulator 5 up and down.
Incorporates an arm motor 19 by a servo motor into the telescopic linear motion arm 6 that expands and contracts in two stages, and the wrist joint W
Is a wrist pitch motor by a servo motor for pitch or roll rotation of an end effector 8 for gripping and picking a cucumber at one end of a wrist pitch joint shaft 7 internally supported near the distal end of a telescoping linear arm 6. 20 or a wrist roll motor 21 are incorporated and are connected to each other.

A waist motor 25, which is a servomotor for horizontally rotating the manipulator 5, is provided in the base 5a on which the shoulder joint S is mounted, in interlocking connection. A plurality of cucumber can be stored on the lower surface of the base portion 5a. The auxiliary storage case 4, which extends to the position of the end effector 8 at the time of picking the cucumber and which can open and close the bottom surface, is driven by a rack and a pinion. A case slide motor 4a is disposed on the upper surface of the storage case 4 and configured.

The manipulator 5 and the auxiliary storage case 4
Is rotatably supported along the inner wall surface of the cutout portion of the work turning cylinder 15 so as to be vertically movable by sliding the slider 22 in the vertical direction. The slide guide 23 is provided in bearing. The base 5a of the manipulator 5 is provided on the slide guide 23 along the outer wall surface of the cutout portion of the work turning cylinder 15 at right angles.
a, and a slide motor 24 by a servo motor for rotating the slider 22 is connected to the lower end of the slider 22 in an interlocking manner, and the visual sensor 1 and the manipulator 5 are moved together with the work turning cylinder 15 by approximately 180. An undercarriage motor 9a for turning the vehicle is arranged at the center position of the work turning cylinder 15 of the undercarriage 9 and connected in an interlocking manner.

The end effector 8 of the manipulator 5
The gripper 8a grips the cucumber, a fruit detection sensor 8b that detects the cucumber, a gripping motor 8c that grips the detected cucumber, a fruit pattern detection sensor 8d that detects the fruit pattern of the gripped cucumber, A fruit pattern detection slide motor 8e for detection and a cutting motor 8f for cutting the detected fruit pattern are provided.

A slider 26 composed of a ball screw or the like for vertically moving a CCD camera 2 having an interference filter 3 rotatably mounted thereon as the visual sensor 1 is provided with:
The work revolving drum 15 is rotatably supported along the inner wall surface of the cutout at a right angle, and a slide guide 27 is provided so as to be vertically movable by the rotation of the slider 26. The engaging arm of the visual sensor 1 passes through a sliding groove 15b provided on the upper surface of the slide guide 27 along an outer wall surface displaced to a position substantially perpendicular to the outer wall surface of the manipulator 5 having the sliding groove 15a. 1a is engaged, and a slide motor 28 by a servo motor for rotating the slider 26 is interlockedly connected to the lower end of the slider 26,
The sliding groove 15b is provided with a guide rail that can slide while keeping the posture of the visual sensor 1.

As a control mechanism, as shown in FIG. 9, a traveling section controller 29a for controlling the traveling apparatus 13, a visual section controller 29b for controlling the visual sensor 1, and a thin section controller 29c for controlling the manipulator 5 are provided.
And a main controller 29 that connects these controllers 29a, 29b, and 29c and controls them in an integrated manner in the front cover 14.

The main controller 29 is connected to the operation panel 17a and the safety device 14a. The traveling unit controller 29a is configured by controllably connecting the wheel drive motor 11, the steering angle sensor 12, the carry control unit 16a, and the like. The visual unit controller 29b includes:
As shown in FIG. 10, a receiver 3 receives a signal from a drive pulse generation circuit 31 via a driver 32 and receives a signal from the OCR 30.
3 from the receiver 33 to the CCD camera 2. A signal processing circuit 34 for converting a signal detected by driving the camera 2 into a video signal is connected to the camera 2,
The signal processing circuit 34 is connected to an image memory 36 via an A / D converter 35, and the image memory 36, an image processing circuit 37 for performing high-speed image processing, and a CPU 38 are communicably connected. The CPU 38 is connected to a timing circuit 39 for setting the lamp emission timing and the start timing of the A / D converter 35 based on the video signal, and the timing circuit 39, the A / D converter 35, and the lamp drive circuit 40 are connected. Connection and the timing circuit 39
, A signal processing circuit 34 is connected, the lamp driving circuit 40 is connected to drive the halogen lamp 41, and the photosensor 47 is connected to the CPU 38 through A / D conversion.

As shown in FIG. 11, the thinning controller 29c includes the shoulder motor 1 as a servomotor.
8, an arm motor 19, a wrist pitch motor 20, a wrist roll motor 21, a slide motor 24, a waist motor 25, and a slide motor 28 are connected via a control driver 42, respectively, and the fruit detection sensor 8b, gripping motor 8c, The pattern detection sensor 8d, the fruit pattern detection slide motor 8e, and the cutting motor 8f are connected to each other. The servo motors 18, 19, 20, 2
Each of 1, 24, 25, and 28 is provided with a combination of an encoder and an origin detection sensor.

As shown in FIG. 1, the visual sensor 1 includes an interference filter 3a on the inner side of the front surface of a cylindrical outer case 43, which can capture a density value image at a wavelength of 550 nm in a visible light region to a near infrared region. A disk-shaped interference filter 3 formed by half-split and an interference filter 3b capable of imaging a density value image at a wavelength of 850 nm is arranged, and the axis of the interference filter 3 is positioned rearward by a filter drive shaft 44. The CCD camera 2 is attached to the lower side of the filter drive shaft 44 in conjunction with the motor 45, and the front surface of the outer case 43 is provided with an imaging hole 43a only in the lens portion of the CCD camera 2.

Above the filter drive shaft 44, a filter position sensor 46 such as a photomicro sensor for detecting the rotational position of the interference filter 3 is disposed.
A photosensor 47 for detecting the intensity of light at the time of image pickup is arranged at the lower part of the outer periphery of the outer case 43, and the halogen lamp 41 as a light source is attached to the upper part of the outer periphery of the outer case 43.

As shown in FIG. 2, the CCD camera 2, the filter motor 45, the filter position sensor 46, the photo sensor 47, the halogen lamp 41 and the like are connected to a camera controller 49 via an interface 48. In order for the agricultural robot to harvest cucumber as a work object, the traveling unit controller 29a controls the wheel drive motor 11 and the steering angle sensor 12 while traveling along the ridge groove R, and the inclined shelf C The cucumber cultivated at the right time of harvest is sent to the visual controller 2
9b and the calculation of the thinning section controller 29c, the harvesting approach is performed by moving the visual sensor 1 and the manipulator 5 up and down and turning left and right. With this approach, the end effector 8 is freely used, and the fruit detection sensor 8b is used.
The gripper 8c is actuated by the detection of the cucumber, the cucumber is gripped by the gripper 8a, and the fruit of the gripped cucumber is detected by the fruit pattern detection sensor 8d and the fruit pattern detection slide motor 8e. Is cut by operating a cutting motor 8f to perform fruit cutting. The picked cucumber is collected and delivered to the carry, and is discharged to the ground by the carry control unit 16a when the carry is full.

On the base portion 5a of the manipulator 5 which can be slid up and down while being connected to the work turning cylinder 15,
The wrist pitch joint shaft 7 at the tip of the telescopic linear arm 6 of the manipulator 5 is engaged with the visual sensor 1 that slides up and down separately from the manipulator 5.
At the axial end of the end effector 8, which rotates about 180 degrees in the vertical and horizontal directions, respectively, and the visual sensor 1.
Since the visual axis of the CCD camera 2 can be made coincident with the above, no misalignment occurs in the direction of the harvest approach.

The image pickup by the visual sensor 1 is performed by rotating the interference filter 3 through the filter driving shaft 44 by driving the filter motor 45, and by this rotation, two kinds of interference filters 3a and 3 having wavelengths of 550 nm and 850 nm.
b alternately passes through the front surface of the CCD camera 2, the passing position is detected by the filter position sensor 46, and the image is transmitted through two types of interference filters 3 a and 3 b which are instantaneously replaced, and two types of density values are obtained. Images can be continuously input.

As shown in FIG. 3, the image processing based on this input image is based on the reflectance at a wavelength of 550 nm.
Both the foliage show a low value of about 20 to 30%, but the reflectance at a wavelength of 850 nm shows a high value of about 80% for cucumber compared to about 50% for the foliage. The value ratio is calculated as: density value (850) / density value (850) + density value (55)
0) The luminance level (25)
An arithmetic image as shown in FIG. 4 is obtained by the arithmetic value multiplied by 5), and the cucumber can be accurately recognized by a binary image obtained by binarizing the arithmetic image.

The two types of interference filters 3a, 3b
The amount of light is measured by the photo sensor 47 in synchronization with the input of an image picked up through the camera, and the presence or absence of an optical difference between the two images is detected, thereby instantaneously replacing the interference filters 3a and 3b. Even if images are input continuously, sunlight may change between the two image inputs, so if there is an optical difference between the two images, the image input is redone, If there is no difference, it is assumed that the image has been input under the same condition, and the process proceeds to the next step to perform the operation between the two images. As described above, if there is an optical change between two image inputs, appropriate image data can be obtained by redoing the input, so that the cucumber can be recognized relatively easily.

In the harvesting operation by the visual sensor 1 and the manipulator 5, as shown in the flowchart of FIG. 12, first, the visual sensor 1 and the manipulator 5 respectively move the position of the initial posture, that is, the image of the inclined shelf C for growing cucumber up and down. The image is divided into two stages (the number of division stages can be set arbitrarily), and an image is input by the visual sensor 1 at the lower position. Only the visual sensor 1 is slid upward in parallel with the image processing for cucumber recognition using the lower input image,
Upon completion of the lower image processing, an image is input by the visual sensor 1 at the upper position.

Upon completion of the input of the upper image, when the cucumber is present in the image area as a result of the lower image processing, the lower harvest approach is performed, and in parallel with the lower harvest approach, the image of the upper input image is used. The processing is performed, and upon completion of the lower harvest, as a result of the upper image processing, if there is a cucumber in the image area, the manipulator 5 is slid upward to perform the upper harvest approach. Cucumbers harvested by the upper and lower harvesting approaches are stored in the auxiliary storage case 4.

When the upper harvesting is completed, the visual sensor 1 and the manipulator 5 are returned to their initial posture positions, and the work turning cylinder 15 is rotated horizontally by approximately 180 degrees. The timing at which the control unit 16a passes above the carry is detected, and the cucumber stored in the auxiliary storage case 4 is stored in the carry by opening the bottom surface of the case 4 by this detection. With the completion of the turning of the work turning cylinder 15, the harvesting operation on the inclined shelf C of the cucumber on the opposite side is performed in the manner described above.

By performing such a harvesting operation, two or three cucumbers are harvested at the peak in the inclined shelf C on one side. Therefore, the cucumbers harvested in the conventional manner are placed one by one into the carry by the manipulator 5. In the case where the cucumber is stored, the cucumber transport time can be reduced to ２〜 to 比 べ as compared with the case where the harvest time is increased only in the cucumber transport process. For this reason, since unnecessary operation is not required, time can be reduced and power consumption can be reduced.

In the visual sensor 1, the engagement arm 1a of the sensor 1 is engaged with the upper surface of the slide guide 27, and the upward movement is held by the guide rail of the work turning cylinder 15. Since it can be moved freely and can be moved downward by its own weight, even if a failure occurs in the manipulator 5 and a runaway occurs as in the case where the visual sensor 1 is fixed to the slide guide 27 as in the related art, the collision occurs. There is no such thing as being damaged. Since it is possible to slide in accordance with the movement of the manipulator 5, the positional relationship between the visual sensor 1 and the manipulator 5 due to collision does not occur, and the work center axis of the end effector 8 of the manipulator 5 and the visual The optical axis of the sensor 1 is accurately realigned (a dedicated jig is required) and there is no trouble. Therefore, it is possible to prevent damage loss due to collision of the most expensive visual sensor 1 among many sensors.

In this embodiment, the object to be harvested is limited to cucumber. However, it is needless to say that the gist of the present invention can be applied to fruits and vegetables other than cucumber.

[Brief description of the drawings]

FIG. 1 is a front view and a side view showing a detailed structure of a visual sensor.

FIG. 2 is a block diagram showing a control relationship of a visual sensor.

FIG. 3 is a diagram showing spectral reflection characteristics of cucumber and its foliage.

FIG. 4 is an image diagram showing a calculation image based on a luminance level.

5A and 5B are a plan view and a side view showing a state in which a telescopic translation arm of the manipulator is extended.

FIGS. 6A and 6B are a plan view and a side view showing a state in which the cucumber is pinched and grasped by the manipulator and the telescopic linear motion arm is shortened.

FIG. 7 is a side view showing the whole of the agricultural robot.

FIG. 8 is a back sectional view showing the whole of the agricultural robot.

FIG. 9 is a block diagram showing a control relationship of the entire agricultural robot.

FIG. 10 is a block diagram showing a control relationship of a visual unit.

FIG. 11 is a block diagram showing a control relationship of a fruit cutting unit.

FIG. 12 is a flowchart showing a harvesting operation procedure of the agricultural robot.

FIG. 13 is a work diagram showing an image pickup state of a lower shelf and an upper shelf by a visual sensor.

FIG. 14 is a work diagram showing a state of picking cucumber at the upper stage of the inclined shelf by the manipulator and an auxiliary storage state.

[Explanation of symbols]

 1. Visual sensor 2. CCD camera 3a. Interference filter 3b. Interference filter

Claims (1)

[Claims] 1. Two types of interference filters 3a and 3b whose image density values change by transmission are provided on the front surface of a CCD camera 2 as a two-wavelength type visual sensor 1 for recognizing a work object.
Are arranged so that they can be replaced instantaneously, and an image capturing apparatus of an agricultural robot that continuously inputs images via the replaced interference filters 3a and 3b.