JPH07222505A - Method for controlling travel of self-traveling working vehicle - Google Patents

Abstract

PURPOSE:To obtain highly reliable boundary information by removing the affection of non- worked targets in an imaged picture image, when the boundary is detected from the imaged picture image for controlling the travel of a self-traveling working vehicle along the boundary between a worked target region and a non-worked region. CONSTITUTION:A green picture image is subjected to a two-dimensional transfer average treatment for the removal of noises and singular points, and the quantities of changed strengths in the horizontal axis direction are subsequently computed to obtain a differential value picture image. Picture elements imaging turf-free targets having color strength ratios excluding a preliminarily set range or having the absolute values of green color strengths in the preliminarily set range are extracted as mask information from the respective picture images of R, G and B. The differential values of the picture elements imaging the targets excluding the turf and their peripheral picture images are rewritten to zero to produce a binary picture image in which the data of the articles excluding the turf are removed from the green differential value picture image. The gradient and intercept of a straight line approximating a working boundary determined by the Hough transformation of the binary picture image. Thereby, the affection of the non-worked targets in the imaged picture image can be removed to obtain high reliable boundary information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention processes a captured image around a vehicle to detect a boundary between a work target area and a non-work target area, and controls a steering system so as to autonomously travel along the detected boundary. The present invention relates to a traveling control method for an autonomous traveling work vehicle.

[0002]

2. Description of the Related Art Conventionally, for an autonomous vehicle that is autonomously autonomously traveling, there is a technique in which an electric wire is buried underground and a magnetic sensor detects a magnetic field generated by the electric wire as self-position detection for autonomous traveling. Although it has been proposed, if the autonomous driving area is large, such as an autonomous vehicle that performs unmanned grass cutting, lawn cutting, etc. in various fields such as golf courses, river banks, parks, etc. It is difficult to embed the electric wire, and the installation cost becomes large.

Therefore, in the above-mentioned autonomous vehicle, a boundary between the unprocessed land and the unprocessed land on the field is detected and detected using a non-contact sensor such as an optical sensor or a mechanical contact sensor. Many of them follow the boundary, and recently, as disclosed in Japanese Patent Laid-Open No. 61-139304, a camera is mounted to capture a peripheral area including a work target, and a captured image is obtained. There has been proposed a technique for detecting the boundary between the work target area and the non-work target area from the amount of change in brightness in.

[0004]

However, in the technique of detecting the boundary between the work target area and the non-work target area from the change amount of the brightness in the captured image when performing the contour traveling along the boundary, the captured image is The amount of change in the brightness of the object other than the work target shown in FIG. 3 is also captured, and the position of the boundary may be erroneously recognized, making it difficult for the copying traveling.

The present invention has been made in view of the above circumstances, and when the boundary is detected from a captured image in order to perform traveling control along the boundary between the work target area and the non-work target area,
It is an object of the present invention to provide a traveling control method for an autonomous traveling work vehicle that can remove the influence of a non-working object in a captured image and obtain highly reliable boundary information.

[0006]

A first aspect of the present invention detects a boundary between a work target area and a non-work target area from a captured image around a vehicle, and controls a steering system so as to autonomously travel along the detected boundary. In the traveling control method of the autonomous traveling work vehicle,
When calculating the amount of change in brightness from the captured image to generate a binarized image, the amount of change in brightness of the pixels that do not satisfy the condition related to the color for identifying the work target and the peripheral pixels thereof is zero. Is generated, and a straight line approximating the boundary is calculated from the binarized image.

A second invention is characterized in that the condition relating to the color for identifying the work object in the first invention is set from an image picked up according to a specific operation input.

[0008]

According to the first aspect of the present invention, the amount of change in the brightness of a pixel which does not satisfy the condition relating to the color for identifying the work target and the peripheral pixel thereof is set to zero in the image of the periphery of the vehicle.
A binarized image is generated, and a straight line that approximates the boundary between the work target region and the non-work target region is calculated from the generated binarized image.

According to a second aspect of the present invention, in the first aspect, a condition relating to a color for identifying a work object is set from an image picked up in accordance with a specific operation input, and a pixel which does not satisfy this condition and its periphery are set. The amount of change in pixel brightness is set to zero and 2
Generate a binarized image.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. 1 is a flow chart of a boundary detection routine, FIG. 2 is a flow chart of a traveling control routine, FIG. 3 is a schematic explanatory view showing an outer appearance of a lawn mowing vehicle, and FIG. 4 is a steering control system. FIG. 5 is a basic block diagram, FIG. 5 is a circuit block diagram of a boundary detection unit, FIG. 6 is an explanatory diagram showing an original image of a work area, and FIG. 7 is an explanatory diagram showing an averaging processed image.
8 is an explanatory diagram showing a differential value image, FIG. 9 is an explanatory diagram showing mask information, FIG. 10 is an explanatory diagram showing a differential value image after masking, FIG. 11 is an explanatory diagram showing a slice level calculation histogram, and FIG. Is an explanatory view showing a binarized image, FIGS. 13 and 14 are explanatory views showing a two-dimensional histogram for Hough transform,
FIG. 15 is an explanatory diagram showing the straight line extraction result, and FIG. 16 is an explanatory diagram showing the recognition result of the boundary approximate straight line.

In FIG. 3, reference numeral 1 indicates an autonomous traveling work vehicle capable of being autonomously driven by an unmanned vehicle. In this embodiment, it is a lawn mowing work vehicle for performing grass / lawn mowing work on a golf course or the like. This lawnmower 1 is provided with a cutting blade mechanism 2 such as a mower for performing grass / lawn mowing work on the lower part of the vehicle body, and as an image pickup means for picking up a work area, for example, a solid-state image sensor (CCD) is used. A CCD camera 3 for color image pickup having a color mosaic filter on the image pickup surface is provided on the front side of the vehicle, and a dead reckoning sensor including a geomagnetic sensor 4 and a wheel encoder 5 is provided for calculating a traveling history.

The lawnmower working vehicle 1 is driven by an engine, and as shown in FIG. 4, the front wheel steering mechanism 9a and the rear wheel steering mechanism 9b are independent by a front wheel hydraulic cylinder 8a and a rear wheel hydraulic cylinder 8b, respectively. And is driven. A hydraulic pump 6 driven by the engine 1a is connected to each of the hydraulic cylinders 8a and 8b via a front wheel steering hydraulic control valve 7a and a rear wheel steering hydraulic control valve 7b, respectively. To be done.

The control device 20 is connected to the boundary detection section 30 to which the CCD camera 3 and a condition setting switch 70 described later are connected, the front wheel steering hydraulic control valve 7a and the rear wheel steering hydraulic control valve 7b. In addition, the traveling control unit 40 to which signals from the steering angle sensors 10a and 10b respectively mounted on the front wheel steering mechanism 9a and the rear wheel steering mechanism 9b, the geomagnetic sensor 4, the wheel encoder 5 and the like are input.
And are provided, each of which is mainly configured by a microcomputer.

That is, the boundary detection unit 30 detects, for example, a boundary between a fairway and a rough on a golf course, a boundary between a worked area and a not-worked area during grass and lawn mowing from a captured image around the vehicle. As described above, the boundary between the work target area and the non-work target area (hereinafter, referred to as a work boundary) is detected. Further, in the traveling control unit 40, the boundary detection unit 3
The steering system is controlled so that the vehicle travels along the work boundary detected at 0.

A concrete circuit configuration of the boundary detecting section 30 is shown in FIG. 5, which includes a CPU 50, a RAM 51 for holding work data, a ROM 52 in which fixed control data and a control program are stored, and a condition setting described later. The data bus 54 and the address bus 55 of the microcomputer provided with the input / output (I / O) interface 53 to which the switch 70 is connected, for example, one frame 512 × 51.
A video memory 67 composed of three frame memories corresponding to the three primary colors of R (red), G (green), and B (blue) light of two pixels is connected via a switching circuit 66.

Further, AD converters 59, 60, 61 and an address control circuit 63 are connected to the video memory 67 via the switching circuit 66, and each AD converter 5 is connected.
Amplifiers 56, 57 and 58 for amplifying R, G and B signals from the CCD camera 3 are respectively connected to 9, 60 and 61, and C is connected to the address control circuit 63.
A synchronizing circuit 62 for generating a timing signal from the synchronizing signal of the CD camera 3 is connected.

In the AD converters 59, 60 and 61, the R, G and B amplified by the amplifiers 56, 57 and 58, respectively.
The signal is sent to the synchronization circuit 6 at a sample rate of, for example, about 8 MHz.
The data is converted into digital data in synchronization with the timing signal from 2 and output to the switching circuit 66 via the data bus 64. Further, the address control circuit 63 uses the synchronization circuit 62.
The address data is generated in synchronization with the timing signal from and is supplied to the switching circuit 66 via the address bus 65.

The switching circuit 66 includes a data bus 54 and an address bus 55 on the CPU 50 side, an AD converter 59,
One of the data bus 64 and the address bus 65 on the 60, 61 side is selectively connected to the video memory 67, and AD is supplied while the address data is being supplied from the address control circuit 63 to the switching circuit 66. Converter 59,
The data bus 64 on the side of 60, 61 is connected to the video memory 67 to write image data, and during this time, access to the video memory 67 by the CPU 50 is prohibited.

Then, R, G, B from the CCD camera 3
When the supply of the signal is stopped and the CPU 50 can access the video memory 67, the CPU 50 reads the image data from the video memory 67, processes the read image data, and inclines and intercepts the straight line approximating the work boundary. Is obtained and then output to the traveling control unit 40 via the I / O interface 53.

The work boundary detection by the boundary detection unit 30 and the copying control by the traveling control unit 40 will be described below.

First, FIG. 1 executed by the boundary detection unit 30.
In the boundary detection routine (1), when the image data is input from the video memory 67 in step S101, the process for the green image and the process for the R, G, B color images are executed in parallel.

That is, in step S102, two-dimensional moving average processing for removing noise and singularity is performed on the green image, and then in step S103, a change point (boundary) of brightness on the screen is extracted. Therefore, the intensity (brightness) in the horizontal axis direction
A process of calculating the amount of change is performed, and in parallel with these processes, a process of calculating mask information for removing objects other than turf from the intensities of the colors R, G, and B is performed in parallel with these processes.

In the two-dimensional moving average processing in step S102, for example, the intensity average value of 20 × 20 pixels is 4 pixels in the X and Y directions with respect to the original image of FIG. This is performed by obtaining the average value of the pixel data in the state of being shifted by one by one, and a green averaging processed image as shown in FIG. 7 is obtained. Then, in step S103, the intensity change amount in the horizontal axis direction is calculated for the averaged image subjected to the two-dimensional moving average process, and a differential image as shown in FIG. 8 is obtained.

WD in FIG. 6 indicates a window, and a straight line approximating the work boundary is obtained in this window by the following processing.

The calculation of the mask information in step S102 'is performed by extracting pixels having a color intensity ratio outside the preset range or a green intensity absolute value within the preset range. Pixels in which objects other than turf are reflected in the green image are identified.

For example, assuming that green intensity, red intensity, and blue intensity are Gi, Ri, and Bi, respectively, the following (1),
Pixels satisfying any of the conditions (2) and (3), that is, as shown in FIG. 9, pixels under the condition (1) in which the red intensity Ri is stronger than the green intensity Gi in the pear-skin display, or the diagonal line display. Of the condition (2) in which the green intensity Gi is smaller than the set value with respect to the blue intensity Bi, and further, the pixel of the condition (3) in which the absolute value of the green intensity Gi in the grove is smaller than the set range is the grass. If not, the extracted pixel is applied to the green image as mask information.

Gi / Ri <1.0 (1) Gi / Bi <1.15 (2) Gi <30% full scale (3) These conditions are the CCD camera 3 in the field.
The image data captured at the moment when the condition setting switch 70 provided outside is turned on, with the
It can be set from each average value of G and B, for example, as a condition according to the actual field situation such as the difference in the green strength of the grass in summer and winter, surely exclude objects other than grass can do.

In this embodiment, the green image is binarized because the work object is grass, but other images may be binarized. For example, R, G,
Instead of the CCD camera 3 which outputs each signal of B, NTS
When a camera that outputs a C-type composite color video signal is used as the imaging means, mask information is calculated from the color information obtained from the composite color video signal, and this mask information is applied to a color image to be binarized. You may get an image.
Further, depending on the characteristics of the color of the work target, it is possible to use two color imaging cameras for obtaining mask information in advance and two monochrome imaging cameras for obtaining a binarized image.

Then, in step S104, based on the mask information, the pixels other than the turf are photographed and the differential values of the peripheral pixels are rewritten to zero to obtain the data other than the turf from the green differential image. When the differential value image as shown in FIG. 10 is eliminated, the process proceeds to step S105, and the slice level for binarizing the masked differential value image is calculated. As shown in FIG. 11A, this slice level is calculated from a histogram created with the intensity differential amount as the horizontal axis and the frequency as the vertical axis, and for example, for the top 10 data of the intensity change amount in the set screen area. It is set to about 50% of the average value.

After that, the process proceeds to step S106, and the pixels exceeding the slice level set in step S105 are
Extraction is performed from the differential value image generated in step S103 to generate a binary image as shown in FIG.
Proceed to S107.

In this case, if the image is binarized using the slice level calculation histogram of FIG. 11B obtained from the differential image of FIG. 8 in which data other than turf is not excluded, FIG. 12B is obtained. As shown in FIG. 6, even the part of the tree or the person who is not the work target, which is reflected at the edge of the screen, is erroneously binarized.

On the other hand, based on the mask information,
11 is obtained from the differential value image of FIG. 10 of only the grass / turf portion in which the non-turf portion between the person or the grove, the leaves and the leaves is excluded, and the leaves themselves of the grove are also excluded as peripheral pixels of pixels other than the grass. In the binarized image of FIG. 12A, which is binarized using the slice level calculation histogram of FIG. 12A, it can be seen that only the grass or lawn that is the work target is appropriately binarized.

Then, in step S107,
A two-dimensional histogram as shown in FIG. 13 is created from the binarized image obtained in S106, and Hough transformation is performed using this two-dimensional histogram to obtain the slope and intercept of a straight line that approximates the work boundary. This two-dimensional histogram has a window WD
Of the window WD in the XY coordinate system in which the lower part of the center is the origin O, the horizontal direction of the screen is the X axis, and the vertical direction of the screen is the Y axis.
The slope tan ^-1 a and the X-axis intercept b when a straight line connecting two pixels included in the pixel are represented by x = ay + b are obtained for all combinations of the two pixels, and the slope tan ^-1 a is the horizontal axis. , The intercept b is plotted on the vertical axis. In the figure, the frequency of the pear-skin portion indicated by Rh is the highest, followed by the halftone line portion indicated by Gh, the shaded portion indicated by Bh, and the other frequency is reduced. The number is zero.

Then, by extracting the peak points from the created histogram, a plurality of straight lines are obtained as shown in FIG. 15 (a), and among these straight lines, the frequency is the highest (the correlation is strong). When the object is extracted, a boundary approximation line L indicating the boundary between the fairway and the rough as shown by the broken line in FIG. 16 is obtained, and the boundary between the fairway and the rough can be accurately recognized.

On the other hand, the two-dimensional histogram obtained from the binarized image generated without excluding the data of objects other than grass is shown in FIG. 14, and the straight line extracted from this two-dimensional histogram is shown in FIG. 15 (b). As shown in, there is a boundary between the fairway and the rough in the forest at the edge of the screen, and it can be seen that it is clearly misrecognized.

After that, when the inclination and intercept of the straight line obtained in step S107 are output to the traveling control unit 40 in step S108, the process returns to step S101 and a new image is input and the above processing is repeated.

As described above, the boundary detection unit is a work boundary having different heights of grass / turf leaves, such as a boundary with a rough road or a boundary between a grass / lawn mowed area and an unworked area. When detected by 30, the traveling control unit 40 controls the traveling along the work boundary by the traveling control routine of FIG.

This traveling control routine is based on previously stored traveling route information such as a type of performing lawn mowing work while reciprocating for each stroke, a type of performing lawn mowing work in a ring shape, the geomagnetic sensor 4 and the wheel encoder 5. This is a basic routine for traveling in accordance with the traveling history calculated based on the signal of
When the boundary approximation straight line L on the screen detected at 0 is input, this straight line L is compared with the preset target straight line L0 (y = px + q) on the screen in step S202,
Deviation amount C from the target straight line L0 of the vehicle body of the lawnmower 1
And the deviation angle θ with respect to the target straight line L0 in the traveling direction.

Next, in step S203, the deviation amount C
And a front wheel target rudder angle and a rear wheel target rudder angle that keep the deviation angle θ constant, and in step S204, signals from the front and rear wheel rudder angle sensors 10a and 10b are input to input the front wheel rudder angle and rear wheel rudder angle. The wheel steering angle is obtained, and in step S205, the front wheel steering angle is compared with the front wheel target steering angle.

As a result of the comparison in step S205, if the front wheel steering angle ≧ the front wheel target steering angle, the process proceeds to step S206, the front wheel steering hydraulic control valve 7a is turned off, and if the front wheel steering angle <the front wheel target steering angle To step S207, the front wheel steering hydraulic control valve 7a is turned on, and the process proceeds to step S208.

In step S208, the rear wheel steering angle is compared with the rear wheel target steering angle. If the rear wheel steering angle ≧ the rear wheel target steering angle, the process proceeds to step S209, and the rear wheel steering hydraulic control valve is operated. 7b
Is turned off, and if the rear wheel steering angle <rear wheel target steering angle, the process branches to step S210 to turn on the rear wheel steering hydraulic control valve 7b.
And

Thereafter, the process proceeds to step S211, and it is determined whether or not a preset time (for example, a control interval of several seconds) has elapsed. If this control interval has not elapsed for the set time, the above-mentioned Returning to step S204, the front and rear wheel steering angles are detected by the signals from the front and rear wheel steering angle sensors 10a and 10b, and the same process is repeated to control the front and rear wheels to converge to the target steering angle. If the interval has exceeded the set time, step S201
The target front and rear wheel steering angles are corrected by returning to and inputting the boundary approximation straight line equation again.

[0043]

As described above, according to the present invention, a color for identifying a work target when a binarized image is generated by calculating the amount of change in brightness from an image picked up around the vehicle. A binarized image is generated by setting the amount of change in brightness of pixels that do not satisfy the condition of 1 above and the surrounding pixels to zero, and a straight line that approximates the boundary between the work target region and the non-work target region Since the calculation is performed, the influence of the non-working object in the captured image can be removed, and highly reliable boundary information can be obtained. Further, at this time, by setting the condition related to the color for identifying the work target from the image captured according to the specific operation input, the condition can be set according to the actual field situation. An excellent effect such as the effect of the non-working object can be surely removed can be obtained.

[Brief description of drawings]

FIG. 1 is a flowchart of a boundary detection routine.

FIG. 2 is a flowchart of a traveling control routine

FIG. 3 is a schematic explanatory view showing the appearance of a lawnmower work vehicle.

FIG. 4 is a basic configuration diagram of a steering control system.

FIG. 5 is a circuit block diagram of a boundary detection unit.

FIG. 6 is an explanatory diagram showing an original image obtained by imaging a work area.

FIG. 7 is an explanatory diagram showing an averaged image.

FIG. 8 is an explanatory diagram showing a differential value image.

FIG. 9 is an explanatory diagram showing mask information.

FIG. 10 is an explanatory diagram showing a differential value image after masking.

FIG. 11 is an explanatory diagram showing a slice level calculation histogram.

FIG. 12 is an explanatory diagram showing a binarized image.

FIG. 13 is an explanatory diagram showing a two-dimensional histogram for Hough transform.

FIG. 14 is an explanatory diagram showing a two-dimensional histogram for Hough transform.

FIG. 15 is an explanatory diagram showing a straight line extraction result.

FIG. 16 is an explanatory diagram showing a recognition result of a boundary approximation line.

[Explanation of symbols]

 1 Lawn mowing work vehicle (autonomous traveling work vehicle) 3 CCD camera (imaging means)

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location G06T 7/00

Claims (2)

[Claims] 1. A travel control method for an autonomous traveling work vehicle, which detects a boundary between a work target area and a non-work target area from an imaged image around the vehicle and controls a steering system so as to autonomously travel along the detected boundary. When calculating the amount of change in brightness from the captured image to generate a binarized image, the amount of change in brightness of pixels that do not satisfy the condition related to the color for identifying the work target and the amount of change in brightness A traveling control method for an autonomous traveling work vehicle, comprising: generating a binarized image as zero and calculating a straight line approximating the boundary from the binarized image. 2. The traveling control method for an autonomous traveling work vehicle according to claim 1, wherein the condition relating to the color for identifying the work target is set from an image captured according to a specific operation input. .