JPH0610774B2 - Imaging type steering control device for automated driving vehicles - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention relates to an unprocessed work site and a processed work site on the front side of travel every time a vehicle body travels a set distance or a set time elapses. An image pickup means for repeatedly picking up an image of a portion corresponding to a boundary in a two-dimensional direction, a boundary position detection means for detecting a position of the boundary with respect to the vehicle body based on image pickup information of the image pickup means, and a boundary position detection means Based on the detection information, based on the target steering angle discriminating means for discriminating the target steering angle of the steering wheel so as to bring the vehicle body closer to the proper state with respect to the boundary, and the discrimination information of the target steering angle discriminating means. The present invention relates to an imaging type steering control device for an automatic traveling work vehicle, which is provided with steering control means for automatically steering the steering wheel to the target steering angle.

[Conventional technology]

The imaging type steering control device of the above-described automatic traveling work vehicle,
Using the imaged information that captured the location of the boundary between the unprocessed work site and the processed work site on the front side of the travel, the vehicle body automatically travels along the boundary. Since it takes image processing that takes processing time to detect the boundary position, it is difficult to continuously detect the positional relationship between the vehicle body and the boundary, which changes as the vehicle body travels. An image is captured every time the vehicle travels or a set time elapses.

However, conventionally, each time the vehicle body travels a set distance or a set time elapses, based on the boundary position information detected based on the imaging information imaged at the imaging point, the steering wheel is changed each time. The target steering angle was determined.

[Problems to be solved by the invention]

By the way, in order to detect the boundary position based on the image pickup information of the image pickup means, for example, the fact that the unprocessed work site and the processed work site appear to have different brightness is used to change the brightness of the captured image. Image processing such as finding an approximate straight line corresponding to the boundary between the unprocessed work site and the processed work site will be performed based on, but the boundary will be interrupted or the brightness of the image part corresponding to the boundary will change. If it is small, there is a possibility that the boundary position may be erroneously detected.

If the boundary position is erroneously detected, the steering operation may be performed in a direction deviating from the original boundary position, the boundary may be out of the imaging field of view, the boundary may be lost, and the vehicle may not be able to automatically travel properly. .

The present invention has been made in view of the above circumstances, and an object thereof is to prevent the boundary from being lost even if the boundary position is erroneously detected.

[Means for solving problems]

The characteristic configuration of the image pickup-type steering control device for an automated work vehicle according to the present invention includes distance information of the vehicle body traveling from one image pickup by the image pickup means to the next image pickup, and target steering of the steered wheels. Boundary position estimation means is provided for estimating the position of the boundary at the time when the vehicle body travels to the position where the image pickup means picks up the image based on the angle information and the detection information of the boundary position detection means. The steering angle determining means sets the boundary position detected by the boundary position detecting means on the basis of the image pickup information of the image pickup means at the next image pickup position relative to the boundary position estimated by the boundary position estimating means. In the case of the above deviation, the point is configured to determine the target steering angle based on the boundary position estimated by the boundary position estimating means, and the operation and effect thereof are as follows. is there.

[Action]

The imaging means repeatedly captures a position corresponding to the boundary between the unprocessed work site and the processed work site on the front side of the travel in a two-dimensional direction each time the vehicle body travels a set distance or a set time elapses. The steering control means is configured to
Since the steering wheel is configured to perform the steering operation to the target steering angle, the range of the boundary position detected based on the image pickup information at the position where the image pickup unit next takes an image is imaged by the image pickup unit. It can be estimated based on the distance information on the traveling distance of the vehicle body from the first image pickup to the next image pickup, the target steering angle information of the steered wheels, and the detection information of the boundary position detection means.

In other words, if the boundary position detected at the time when the vehicle body moves to the next imaging position deviates from the boundary position estimated by the boundary position estimating means by the set range or more, the detected boundary position is incorrect. That is, the estimated steering wheel position information is used to determine the target steering angle of the steering wheel.

〔The invention's effect〕

Therefore, the boundary position detected at the time when the vehicle body moves to the next imaging position is accurately judged whether it is wrong, and if the detection is wrong, based on the estimated boundary position information. Since the target steering angle can be discriminated, even if the boundary position is erroneously detected, it is possible to keep track of the boundary.

〔Example〕

 Embodiments of the present invention will be described below with reference to the drawings.

As shown in FIGS. 6 to 8, the boundary between the uncut land (B) as the unprocessed work land and the already cut land (C) as the processed work land.
(L) is an image sensor (S ₁ ) as an image pickup means for taking a two-dimensional image of the work area on the front side of the traveling, in the front part of the vehicle body (V) of the work vehicle for lawn mowing automatically traveling along But the boundary
It is provided so as to capture an image of a portion corresponding to (L) from diagonally above.

In FIG. 7, (1) is a front wheel, (2) is a rear wheel, (3) is a lawn mowing device, and (H) is a steering handle for boarding operation.

However, the front wheel (1) and the rear wheel (2) are so-called four-wheel steering type and four-wheel drive type so that both of them function as steering wheels and driving wheels.

If the image pickup field of view of the image sensor (S ₁ ) is further explained, as shown in FIG. 6, the vehicle body (V) has a boundary () between the uncut land (B) and the already cut land (C). In a state along an appropriate state with respect to (L), the boundary (L) is the widthwise center of the imaging field of view (A) on the ground surface of the image sensor (S ₁ ) in the direction along the vehicle body traveling direction. Reference line passing toward (L
It is designed to be in agreement with ₀ ).

That is, as shown in FIG. 8, the work vehicle has a work area that extends from one side of the square uncut land (B) surrounded by the already cut land (C) to the opposite side as one work stroke. The image sensor (S) so that the vehicle body (V) automatically travels along the boundary (L) between the uncut land (B) and the already cut land (C) on the side along the length direction of the working stroke. ₁ ) Based on the position information of the boundary (L) detected from the imaging information of ₁ ), the steering is to be controlled, and the end portion of one work process, that is, the opposite side of the uncut land (B). As it reaches, the lawn mowing work in a predetermined range is automatically performed by repeating the automatic turning toward the starting end of the next work stroke in the direction intersecting with the work stroke. It will be.

However, in each of the work strokes, the front wheel (1) and the front wheel (2) are steered in opposite phases while the two wheels are steered by steering only the front wheel (1). The four-wheel steering type is used for traveling.

A control configuration for automatically driving the work vehicle will be described. As shown in FIG. 1, the image sensor (S
₁ ) means for detecting the position of the boundary (L) from the imaged information, and based on the detected boundary position information, the vehicle body (V) automatically travels along the boundary (L). A control device (10) utilizing a microcomputer, which constitutes each of the control means, is provided.

In the figure, (4) is a hydraulic cylinder for steering the front wheel (1),
(5) is a hydraulic cylinder for steering the rear wheel (2), and (6) is a hydraulic continuously variable transmission that shifts the output from the engine (E) to drive the front and rear wheels (1), (2). The device is configured so that it can be switched between forward and backward movement and can be changed forward and backward. (7) is a speed change motor, (8) is a control valve for the front wheel hydraulic cylinder (4), (9) is a control valve for the rear wheel hydraulic cylinder (5), (S ₂ )
Is the vehicle body (V) based on the output speed of the transmission (6).
A distance sensor for detecting the traveling distance of (R ₁ ) is the front wheel (1)
Steering angle detection potentiometer that detects the steering angle (θ) of (R ₂ ) is the steering angle detection potentiometer for the rear wheels, and (R ₂ )
₃ ) is a potentiometer for vehicle speed detection that indirectly detects the vehicle speed (v) based on the operating state of the transmission (6).

That is, by using the control device (10),
Of the image sensor (S ₁ ) Based on the imaging information
Boundary position detection for detecting the position of the boundary (L) with respect to the body (V)
Output means (100), the boundary position detection means (100) detection information
Based on the proper state of the vehicle body (V) to the boundary (L)
The target operation of the front wheel (1) as a steering wheel
Target steering angle determination means (101) for determining the steering angle (θf),
Based on the discrimination information of the target steering angle discrimination means (101)
Automatically steer the wheel (1) to the target steering angle (θf)
Steering control means (102) and the image sensor (S ₁ )But
The vehicle body (V) runs from one image to the next.
Distance information, target steering angle information of the front wheel (1), and
Based on the detection information of the boundary position detection means (100),
Image sensor (S ₁ ) To the position where the next image is taken
Boundary for estimating the position of the boundary when (V) travels
Each of the position estimation means (103) will be configured.

Next, the operation of the control device will be described based on the flow chart shown in FIG.

First, a boundary position detection process for detecting the position of the boundary (L) with respect to the vehicle body (V) is performed based on the image pickup information by the image sensor (S ₁ ), and the front wheel (1) is detected based on the detection information. The target steering angle calculation process for obtaining the target steering angle (θf) is performed, and the front wheel detected by the front wheel steering angle detection potentiometer (R ₁ ).
The steering control process of steering operation is performed so that the steering angle (θ) of (1) becomes the target steering angle (θf), and
(V) should be in a proper state with respect to the boundary (L).

That is, the process of performing the steering operation so that the steering angle (θ) of the front wheel (1) becomes the target steering angle (θf) is the steering control means.
It corresponds to (102).

However, each of the steering angle (θ) of the front wheel (1) and the target steering angle (θf) has a value corresponding to a steering neutral state in which the vehicle body (V) is in a straight traveling state, and the right value is set to zero. The case of steering to the left is set as a positive value, and the case of steering to the left is set as a negative value.

Next, based on the detection information of the distance sensor (S ₂ ), it is determined whether or not the end of the work stroke is approached by comparing with a reference value set based on the length of the work stroke.

When the end of the work stroke is not approached, it is determined whether or not the vehicle has traveled the set distance, and the processing after the boundary position detection processing is repeated as the vehicle travels the set distance.

That is, every time the vehicle body (V) travels the set distance, the image pickup process by the image sensor (S ₁ ) is performed, and the boundary position is detected based on the image pickup information.

When approaching the end of the work process, the end detection processing for detecting the end position with respect to the current vehicle body position is performed by using the image pickup information of the image sensor (S ₁ ) and the boundary position detection processing. Based on the detection information, it is determined whether the end has been reached.

As the end reaches, the vehicle body (V) is turned by about 90 degrees toward the starting end of the next work stroke as described above.

As the turn is completed, it is determined whether or not the work is completed based on the number of turns corresponding to the preset number of work strokes, and if the work is completed, the vehicle body (V) is stopped. To finish the work.

However, when the work is not completed, the processes after the boundary detection process described above are repeated.

Next, each process will be described in detail.

First, the boundary position detection processing will be described with reference to the flowchart shown in FIG.

The boundary position detection process described below is performed in the uncut area.
The position of the boundary (L) is detected by utilizing the phenomenon that the cut land (C) looks brighter than that of (B).

That is, the imaging process is performed every time the vehicle body (V) travels a set distance, and based on the imaging information of the image sensor (S ₁ ), it corresponds to a preset pixel density of 32 × 32 pixels. The brightness level of each pixel is quantized.

Next, the process of obtaining the absolute value of the value obtained by differentiating the brightness in the x-axis direction on the image as the differential value based on the value of each of the 8 neighboring pixels adjacent to the periphery of the pixel to be processed is the two-dimensional direction. Is performed for each pixel lined up with.

However, when using this boundary position detection processing to detect the end of the work stroke, the brightness change between the uncut land (B) and the already cut land (C) is in the front-back direction. , The differential value of the brightness in the y-axis direction intersecting the x-axis is obtained.

Then, by extracting the pixels whose differential value of each pixel is equal to or more than a preset threshold value, the pixels whose brightness change is equal to or more than the preset value are extracted, and the image information is binarized.

After binarizing the image, a plurality of straight lines passing through the extracted pixels and having inclinations set in a plurality of steps are obtained by using Hough transform.

However, in the Hough transform, a plurality of straight lines passing through the extracted pixels are set to polar coordinates set in advance in a plurality of stages in the range of 0 to 180 degrees with respect to the x-axis based on the following formula (i). Inclination (θ) with respect to the x-axis as the reference line in the system
And a distance (ρ) from the origin, that is, the center of the screen (x = 16, y = 16) (see FIG. 9).

ρ = x · cos θ + y · sin θ (i) Then, for one pixel, the frequency of each straight line obtained is counted until the value of the inclination (θ) set in the plurality of steps reaches 180 degrees. After repeating the process of adding the two-dimensional histograms, the frequencies of a plurality of types of straight lines passing through all the extracted pixels are counted for each extracted pixel.

After the frequency of straight lines for all the extracted pixels is completed, a combination of the inclination (θ) and the distance (ρ) from the origin, which is the maximum frequency, is obtained from the value added to the two-dimensional histogram. One straight line (Lx) having the maximum frequency (see FIG. 9) is determined by, and the straight line (Lx) is defined as a straight line corresponding to the boundary (L) on the imaging surface of the image sensor (S ₁ ). Will ask.

Next, the image sensor (S ₁ ) on the ground surface measured in advance
Information on the shape and size of the imaging field of view (A) of the
Based on the position (a, b, c) of the pixel (see FIG. 9) on the imaging surface through which (Lx) passes, correction is made to a straight line corresponding to the boundary (L) on the ground surface.

That is, as shown in FIG. 6 and FIG.
The length of the front and rear two sides of the imaging field of view (A) that intersects with (L)
(l ₁ ), (l ₃₂ ), and the pixel position (x = 1
6, y = 16) the length of the imaging field of view (A) in the width direction
(l ₁₆ ) and the distance (h) between the front and rear sides are measured in advance and stored in the control device (5).

Then, based on the values of the distance (ρ) from the origin and the inclination (θ) in the polar coordinate system obtained as the straight line (Lx) having the maximum frequency, the straight line (Lx) having the maximum frequency is the imaging field of view. A part (a, b) (y = 1, where it intersects with two sides before and after (A))
The value of the x coordinate of the pixel located at the position ₃₂ ) (X ₁ , X ₃₂ ),
Also, the value (x ₁₆ ) of the x coordinate at the center of the imaging visual field (the position where y = 16) is obtained from the following equation (ii) which is a modification of the above equation (i).

However, i = 1,16,32.

That is, the coordinate value on the x-axis (X ₁ ,
X ₁₆ , X ₃₂ ) is detected as position information of the boundary (L), and a series of values for obtaining the coordinate value (X ₁ , X ₁₆ , X ₃₂ ) on the x-axis The processing is performed by the boundary position detecting means (1
It will correspond to 00).

Next, based on the flow chart shown in FIG. 4, the boundary position at the next imaging point is estimated based on the boundary position information detected by the boundary position detecting means (100), and at the same time as the steering wheel. A target steering angle calculation process for obtaining the target steering angle (θf) of the front wheel (1) will be described.

That is, the boundary position information (Xi (t-1)) detected based on the image pickup information at the previous image pickup point and the target steering of the front wheel (1) calculated by the target steering angle calculation process described later. Based on the vehicle angle (θf) and the vehicle speed (v) detected by the vehicle speed detecting potentiometer (R ₃ ), the following (iii) formula corresponds to the boundary (L) at the current imaging point. The coordinate value on the x-axis where the maximum frequency line passes through the field of view (A) on the ground
Estimated boundary position corresponding to the value of (X ₁ , X ₁₆ , X ₃₂ ). Is calculated (see FIG. 5).

However, i = 1,16,32 and j = 1,2,3. Kj is a coefficient. Also, t indicates that it is the information of the imaging point this time,
t-1 indicates that it is the information of the previous imaging point.

That is, from the above equation (iii), the estimated boundary position at this imaging point is The processing for obtaining the value corresponds to the boundary position estimating means (103).

Next, the coordinate values (Xi (t)) at the three points on the x-axis detected based on the imaging information of this time and the estimated boundary position obtained by the equation (iii) above. The absolute value of the difference between and the set value (Δj: j = 1, 2, 3; see FIG. 5) is compared for each coordinate value, and the set value (Δj) is compared. If there is more than the above, it is judged that the detected boundary position information is incorrect, and the detected boundary position is detected.
Estimated boundary position corresponding to (Xi (t)) Replace each of.

The detected boundary position (Xi (t)) is set to the estimated boundary position. Of the detected boundary position after replacement
In order to use (Xi (t)) for the next boundary position estimation, the previously detected boundary position (Xi (t-1)) is replaced.

However, when it is less than the set value (Δi), the estimated boundary position The detected boundary position (Xi (t)) is used as it is without being replaced with.

After the detection boundary position (Xi (t)) is determined, the position (δ) as a deviation in the width direction with respect to the reference line (L ₀ ) on the ground surface is calculated based on the following equation (iv). At the same time, the inclination (ψ) with respect to the reference line (L ₀ ) is calculated based on the following equation (v) (see FIG. 6).

The values of the distance (δ) and the inclination (ψ) are the values of the front wheels.
Corresponding to the sign of the target steering angle (θf) in (1), the state where there is no deviation with respect to the reference line (L ₀ ) is set to zero, and the case where there is a deviation to the right is a positive value, and to the left. If they are deviated, they will be set to negative values, respectively.

Then, based on the inclination (ψ), the distance (δ), and the detected steering angle (θ) detected by the steering angle detection potentiometer (R ₁ ) for the front wheels, the following (vi) The target steering angle (θf) is calculated from the equation.

θf = K ₄ · δ + K ₅ · ψ + K ₆ · θ (vi) where K ₄ , K ₅ , and K ₆ are preset coefficients corresponding to the characteristics of the control response in the steering system.

That is, the inclination (ψ) with respect to the reference line (L ₀ ), the distance (δ) in the lateral width direction at the center of the imaging visual field (the position where x = 16, y = 16), and the steering angle detection potentiometer (R
_Based on the detected steering angle (θ) detected in ₁ ),
The process of obtaining the target steering angle (θf) from the equation (vi) corresponds to the target steering angle determination means (101).

[Other embodiment] In the above embodiment, the case where the vehicle body (V) travels a set distance and the imaging process is performed to detect the boundary position is exemplified. The position may be detected.

Further, in the above-mentioned embodiment, in order to detect the boundary position based on the imaging information, the case where the straight line corresponding to the boundary (L) is obtained by using the Hough transform has been exemplified. Boundary position detection means (1
The specific configuration of 00) can be changed in various ways.

Further, in the above-described embodiment, the boundary position at the next imaging point is estimated based on the coordinate values along the x-axis direction at the front and rear ends of the ground surface of the imaging field of view detected at the previous imaging point. Although the case has been illustrated, for example, the position of the maximum frequency straight line (Lx) on the screen or an equation indicating the straight line may be estimated.

Further, in the above embodiment, the case where the next boundary position is estimated based on the target steering angle (θf) is illustrated, but the boundary position is calculated based on the steering angle (θ) of the front wheel (1). May be estimated, and the specific configuration of the boundary position estimating means (103) can be changed in various ways.

Further, in the above-described embodiment, the lateral distance (δ) and inclination (θ) of the detection boundary with respect to the reference line (L ₀ ) set as a state in which the vehicle body (V) is in a proper state with respect to the boundary (L). ) And the target steering angle (θf) is shown as an example, but the target steering angle (θf) is calculated based on only one of the distance (δ) and the inclination (θ). May be requested. Further, when the distance (δ) or the inclination (ψ) is deviated by more than a set value, it may be provided in a direction for correcting the deviation so as to steer at a constant steering angle set in advance. The specific configuration of the target steering angle determination means (101) can be changed in various ways.

Further, in the above embodiment, the case where only the front wheel (1) is steered is illustrated, but for example, the distance (δ) in the lateral width of the detection boundary with respect to the reference line (L ₀ ) is set to the front and rear wheels ( 1), (2)
Is corrected by the parallel steering system that steers in the same phase,
Further, the inclination (ψ) may be corrected by a four-wheel steering system in which the front and rear wheels (1) and (2) are steered in opposite phases, and the steering control means (102) has various specific configurations. Can be changed.

Further, in the above-described embodiment, the present invention is applied to the work vehicle for lawn mowing, and the case where the vehicle is automatically run in the swivel cutting type is illustrated, but the present invention is applicable to various work vehicles. The shape and boundaries of untreated and treated work sites
Various modifications can be made to the specific configuration such as the configuration (L), the traveling configuration, and the configuration of each part of the work vehicle.

It should be noted that reference numerals are added to the claims for convenience of comparison with the drawings, but the present invention is not limited to the structures of the accompanying drawings by the entry.

[Brief description of drawings]

The drawings show an embodiment of an image pickup type steering control device for an automated guided vehicle according to the present invention. Fig. 1 is a block diagram of a control configuration, Fig. 2 is a flowchart of control operation, and Fig. 3 is a boundary position detection process. Of FIG. 4, FIG. 4 is a flowchart of the target steering angle calculation process, FIG. 5 is an explanatory view of the estimated boundary position range, FIG. 6 is a plan view of the imaging visual field, FIG. 7 is the same side view, and FIG. FIG. 9 is an explanatory diagram of the work site, and FIG. 9 is an explanatory diagram of Hough conversion. (1) …… Steering wheel, (V) …… Body, (B) …… Untreated work site,
(C) …… Processed work site, (L) …… Boundary, (S ₁ ) …… Imaging means, (θf) …… Target steering angle, (100) …… Boundary position detection means, (101)… Target steering angle determination means (102) Steering control means (103) Boundary position estimation means.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Katsumi Ito 64, Ishizukita-machi, Sakai City, Osaka Prefecture Kubota Iron Works Co., Ltd. Sakai Works (56) Reference JP 62-241012 (JP, A) JP 61 -15606 (JP, A) JP 60-184309 (JP, A) JP 62-122507 (JP, A) JP 62-162113 (JP, A) JP 62-140113 (JP, A) ) JP-A-61-187706 (JP, A)

Claims (1)

[Claims] 1. A boundary (L) between an unprocessed work site (B) and a processed work site (C) on the front side of travel each time the vehicle body (V) travels a set distance or a set time elapses. imaging means for imaging repeatedly over the two-dimensional directions of a portion corresponding to the (S _1), detecting the position of the boundary (L) with respect to the vehicle body (V) based on the imaging information of the image pickup means (S ₁₎ Boundary position detection means (100)
Based on the detection information of the boundary position detection means (100), the target steering angle (θf) of the steering wheel (1) is determined so that the vehicle body (V) approaches the proper state with respect to the boundary (L). Target steering angle discriminating means (101) and the steering wheel (1) is automatically steered to the target steering angle (θf) based on the discrimination information of the target steering angle discriminating means (101). An image pickup type steering control device for an automatic traveling work vehicle provided with a steering control means (102) for
Distance information of travel of the vehicle body (V) from _one image pickup means (S ₁ ) to the next image pickup, target steering angle information of the steered wheels (1), and the boundary position detection means. Based on the detection information of (100), a boundary position estimating means (103) for estimating the position of the boundary at the time when the vehicle body (V) travels to the position where the image capturing means (S ₁ ) next captures an image is provided. The target steering angle discriminating means (101) is configured to detect the boundary position detecting means (1) based on the imaging information of the imaging means (S ₁ ) at the next imaging position.
The boundary position detected by (00) is the boundary position estimating means (102).
If it deviates by more than a set range from the boundary position estimated by, the target steering angle (θf) is determined based on the boundary position estimated by the boundary position estimating means (102). An image pickup-type steering control device for an automated traveling vehicle configured as described in 1.