JPH02120910A - Image processor for moving vehicle - Google Patents

Abstract

PURPOSE:To narrow down an image processing area and to rapidly execute image processing for recognizing the end of a traveling road by extracting the image processing area for recognizing the end of the traveling road from a picked-up image. CONSTITUTION:A forecasting part 10b interpolates (forecastes) parameters rhop, thetap for regulating the road end of a current image in accordance with an interpolation formula formed by an interpolation formula forming part 10d and a correction part 10c outputs data [R, THETA] relating to a searching range in the image based upon the parameters rhop, thetap to an image extracting part 10a. The image extracting part 10a sends image data in the extracted area to the correction part 10c and the correction part 10c outputs parameters rhoc, thetac for regulating an approximate straight line expressing the road end to a route guiding part 11. Thus, the image processing area can be narrowed down and time required for recognizing the end of the traveling route can be shortened.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to an image processing device for a moving vehicle that recognizes the edge of a running road from an image of the running road.

(Prior Art) In a mobile vehicle that recognizes the outside world and performs autonomous driving control, a tangled edge or the like is generally detected and running control is performed along this tangled edge. For example, JP-A-61-240
As in No. 307, an image is captured, the point at infinity of the tied end is recognized from this image, and the running of the moving vehicle is controlled based on the line at infinity defined from this point. That is, for running control, it is necessary to know at least the error difference in lateral displacement (distance from the end of the road) with respect to the target course.

(Problem to be solved by the invention) By the way, in order to recognize the road edge from an image, the seventh
As shown in the figure, the straight line equation of the running end image y=kx
It is necessary to find +n. This straight line is recognized as being in contact with the edge of the road. However, in the past, in order to obtain this equation, the image data of the entire screen was processed, which took time to recognize the road edge and was not suitable for high-speed processing.

Therefore, the present invention was proposed in order to eliminate the drawbacks of the above-mentioned conventional example, and its purpose is to narrow down the image processing target area in some way and perform image processing for road edge recognition at high speed. The aim is to propose an image processing device for moving vehicles that can perform the following tasks.

(Means and operations for achieving the object) The configuration of the image processing device for a moving vehicle according to the present invention for achieving the above object is such that a moving vehicle that performs driving control based on the recognized road edge is configured to Based on an imaging means that takes an image of the running road for recognition and multiple recognition results regarding the running road edge,
The present invention is characterized by comprising means for extracting an image processing target area for recognizing a running road edge from the captured image, and means for recognizing a running road edge from information on the extracted image area.

(Example) An example in which the image processing device of the present invention is applied to road edge recognition for travel control will be described below with reference to the accompanying drawings.

<Principle of Embodiment> This embodiment focuses on the point that the structure of a road changes continuously and smoothly, especially in an environment such as an expressway. In other words, the likely range of the current or future road edge is predicted from the information of the road edge recognized in the past, and the area within this predicted range in the currently obtained image is used for road edge recognition. The search range is If the road structure changes continuously and smoothly, the possibility that the road edge will be within such a predicted range is quite high. By concentrating image processing within this narrowed range, the speed of road edge recognition can be improved.

The equation of the straight line representing the edge of the road can basically be defined by two parameters. These parameters are, for example, (1) in the XY orthogonal coordinate system, the slope and the intercept (in the case of Figure 7, the slope k and the coordinate position of the intercept 1 or 2 points, and (2), and in the curved coordinate system, , when one point on the straight line is given, the distance ρ between this - point and a predetermined origin, and the intersection angle θ between the line segment connecting this origin and that one point and the coordinate axis.

In addition, there are various ways to express straight lines. in this way,
Since a straight line in a plane is defined by two parameters, in this embodiment, the parameters of straight lines recognized in the past are extrapolated to the present or future to predict approximate parameters (with a range), The currently obtained image is modified using the predicted parameters as initial information, that is, the image area is narrowed down. Then, a straight line representing the edge of the road is quickly and accurately obtained from the narrowed down images. That is, in this embodiment, (i) prediction of parameters based on past recognition results, (ii) setting of search range based on prediction, (iii)
Tangled end recognition is performed in a cycle of: image extraction based on the search range, (iv) parameter calculation based on the image data of the extracted portion, and () updating of recognition results.

<Configuration of Example Apparatus> FIG. 2 is a diagram showing the configuration of the mobile vehicle 1 when the image processing device according to the present invention is incorporated into the mobile vehicle to perform travel control. The moving vehicle 1 includes a video camera 2 for taking images of the outside world, a travel controller 5 incorporating an image processing device, an encoder sensor 3 for detecting the number of rotations of the wheels to measure travel distance, and the actual moving vehicle. The vehicle is equipped with a vehicle controller 4 for controlling the running of the vehicle. This mobile vehicle 1 is driven by a normal gasoline engine (not shown) and is steered by a steering mechanism (not shown). The vehicle controller 4 controls the speed of the vehicle by controlling a throttle actuator, the direction of travel of the vehicle by controlling a steering actuator, and the direction of travel of the vehicle by controlling a brake actuator (not shown). Perform braking.

Note that 6 is a sensor that detects the angle (α) at which the camera 2 is attached, and this angle and the camera attachment position d are used to estimate the horizontal line in the image.

Details of the travel controller 5 are shown in FIG. As shown in the figure, this controller 5 includes an image processing section 10 and a route guidance section 11. The image processing unit 10 includes an image extraction unit], Oa, a prediction unit 1ob, and a correction unit 10c.
It consists of an interpolation formula generation section 10d. The prediction unit 10b generates a parameter ρ1. which will define the tangled edges of the current image, according to the interpolation formula generated by the interpolation formula generation unit 10d (this was generated based on past data). Interpolate (i.e., predict) θ. This prediction method is based on Lagranche (
Lagrange's interpolation method. Prediction part ta
b outputs this prediction parameter ρPθ2 to the correction unit 10c. The correction unit 10c adjusts this parameter ρ2. Based on θ, data [R, θ] regarding the search range within the image is output to the image extraction unit 10a. The image extraction section 10a sends the image data of the extracted area to the modification section 10c. Correction part 10
c is a parameter ρ that defines an approximate straight line representing the tied end using Hough (1-1ough) transformation, which will be described later. θC
Output. This tangled end information parameter ρ. θ. is sent to the route guidance section, which calculates the steering angle, vehicle speed, etc. for running control based on the error between the own army's position and the tangled end, and outputs it to the vehicle controller 4.

In this example, the straight line equation that defines the tangled ends is
We adopt a method expressed by parameters ρ and θ. This is mainly because the parameters ρ and θ are easy to use in the Hough transform. In addition, the Hough transform can be performed faster to obtain a linear equation than the least squares method.

<Hough transform> The Hough transform will be explained according to FIG.

When a tangled edge is manually input as an image, it becomes a collection of 1" pixels. When these "1" pixel points are distributed linearly, the Hough transform approximates a straight line along each of these points. This is the way to get it.

In the XY space, as shown in Figure 3A, two points A: (
Suppose there is xA yA ) B: (Xa ye). Let these points represent "1°° pixels. Any straight line passing through point A is ρ = X AC
O3θ+y As1nθ Also, any straight line passing through point B is expressed as p''X acO3O+y, sinθ.

If we consider ρ and θ as variables, the straight line group passing through point A is the 3rd B
The trigonometric function curve I in the figure will be the same, and the straight line group passing through the point B will be the curve II in FIG. 3B. ρ2 representing a straight line passing through points A and B. θ should be at the intersection of curves ■ and II. That is, ρ2. For θ2, /)P=XACQ3θ,+yAsinθP9 p”
X ecO8θp + yesjnθP holds true. Now, a straight line passing through two points is uniquely determined, but a straight line passing through three points is generally determined only approximately. Three 90 curves corresponding to straight line groups passing through the three points intersect at the three points. If linear approximation is possible to the extent that these three points can be placed on substantially one straight line, the three intersection points should be close to each other. Developing this idea, a 1° pixel group in an image has 9 points on the pθ plane corresponding to each of the straight lines passing through each pixel.
It can be converted to 0 curve. Then, the pixel group is linearly approximated by a straight line specified by a specific ρ "θ" in the distribution of intersection points between these curves, for example, p :XACO3θ"+yAsine".

However, it is not realistic to perform the process of finding an approximate straight line using Hough transform for all points in an image. This is because there are images other than running ends in the image, so calculate the approximate straight line that passes through all pixel points, including pixels that are unrelated to the ends, and use this to approximate the ends as a straight line. This is because that in itself is meaningless.

Therefore, in this embodiment, the process of obtaining an approximate straight line by Hough transform is performed as follows. That is, according to a prediction based on an interpolation formula described later, a predicted value ρ2. θP (P is a subscript indicating prediction) is obtained, and based on this predicted value, a certain range is set as the search target range at the end of the road. Here, in this embodiment, the interpolation formula is based on the Lagranche interpolation method, for example. In addition, the above-mentioned certain range is
defined by the set [R, θ] of ρθ, and RE[ρ, −△
ρ, ρ2+Δθ] θG[θ, −Δθ, θ2+ΔθJ. By such prediction, the conversion is limited to the range Ill as shown in FIG. 3B, and the straight line extraction process is accelerated.

Lagranche interpolation method> The Lagranche interpolation method will be explained based on FIG. This interpolation method is a method of interpolating the next data based on n pieces of past data, and is represented by an interpolation polynomial at 0 o'clock.

FIG. 4A and FIG. 4B show that the moving vehicle is at the traveling position doΦd,
It represents the change in the parameter ρθ obtained in the past when moving sequentially as c=>d2e=>...φd. For example, position d. When the straight line of the tangled end (ρ. θ
. ) is obtained, and when you are at position d+, the straight line (p+
01) was obtained, and...(ρ.On) was obtained. Then, the purpose is to predict ρθ at the traveling position d from these data and obtain pp(a) and pP(dl. Note that d is the encoder sensor 3
obtained from.

Since the prediction method for pp and pP is the same, prediction for ρ will be explained below.

In Figure 4A, the points (do,)O), (at ρ+
)......, (d, ρ.), and the predicted value ρP(d
) is pP(d) a o (d-dl) (d-d2) (d-d3)
-(d-do) ten a+ (d-do) (d-cjz)
(d-d3) −(d-do)+ ao(cldo
) (d-dl) (d-d2) ・= (d-do-+
). Here, (n+1) a. ~an are the coefficients of this polynomial. Point (doρo).

Since (d, ρl), . . . , (dnp, ,) are past points, their values are known. Also,
Since pP(d) in the general formula given by the above equation passes through n+1 points, pp(do)=p.

It is expressed as pp(at)=ρ. pp(d) can be obtained in exactly the same manner. For the sake of distinction, let b represent the coefficient for pP(d), then pP(dn)=ρ.

Therefore, by solving these n+1 equations, the coefficient a
. ~an is calculated, and pp(a) when predicted at point d is therefore,
pP(d) when predicted at point d is.

That is, past data d O ~ d n , a O□ a
n, bo~bn, ρ0~ρ. , 00~θ. If we know the current position d, we can predict pP(d) and pP(d).
As, the aforementioned search range [R, C)], RE[ρ2−△
ρ, ρ2+Δρ] θC[θ2−Δθ, θ2+Δθ].

<Control Procedure> FIG. 5 is a flowchart showing a processing control procedure in the image processing section 10. In step S2, encoder 3
Find the traveling distance d from . In step S4, past data d o ”□ d n , a o to a,, 1.-
bo, ρ. ~ρ. , Oo~θ. without reading it from memory. These past data are stored in a predetermined memory in a readable index manner as shown in FIG. In step S6, interpolation polynomials ρp(a) and pP(d) are generated, and in step S8, new coefficients a' and b' (al etc. in FIG. 6) are calculated, and pp( d)p
Calculate p(a) by g1. In step SIO, search ranges [R, e], Rε[ρP−Δρ, ρP+Δρ] θG[θ2−Δθ, θ2+ΔO] are determined. At this time, △p and △θ are, for example, △ρ2
Ipp-ρ. It is best to choose 1/2 △θ= 108-〇o1/2. This is because po and θ0 are past data used for prediction, and are not values that are different from the predicted values.

In step S12, this search range is sent to the extraction unit 10a to extract image data in that range, and in step S14, based on the image data, an approximate straight line pc=xlcosθC+y+Si
nθ.

Calculate. In step S16, ρ is sent to the vehicle controller. , θ0.

In step S16, for the next prediction, a', b', calculated this time are moved to the a, b area and updated (
(See Figure 6). Similarly, ρ n ′−pc θ. −〇.

and update it.

The above control procedure is repeated every time one image is input.

In this way, the image processing area can be narrowed down by prediction, and the parameters defining the straight line representing the bound end can be calculated at high speed. Then, high-speed travel control becomes possible from the fast-recognized binding edge.

In the above description, the interpolation polynomial is n-th degree, but in view of the balance between prediction accuracy and calculation time, it is appropriate that n=3 to 5. In addition to the above definitions, Δρ and Δθ may be determined in advance according to prediction accuracy.

In the above embodiment, the Hough transform was used for straight line extraction, but this was used because of the calculation speed; an edge detection differential method may also be applied. Further, as the interpolation formula, a Newton interpolation polynomial may be used in addition to the Lagranche polynomial. That is, the present invention is not limited to the straight line expression method, straight line extraction method, or prediction method as long as the search range for straight line extraction is predicted from past data and straight lines are extracted within this range. be.

(Effects of the Invention) As explained above, the configuration of the image processing device for a moving vehicle according to the present invention is such that a moving vehicle that performs running control based on the recognized edge of the running road uses an image of the running road for recognition of the outside world. an imaging means for taking a picture of the roadside; a means for extracting an image processing target area for roadside recognition from the captured image based on a plurality of recognition results regarding the roadside; and information on the extracted image area. The present invention is characterized by comprising means for recognizing the edge of the road on which the vehicle is traveling.

Therefore, since the image processing target area is narrowed down, the time required to recognize the road edge is shortened. In addition, since the image processing target area is defined based on past recognition results,
Even with such narrowing down, there is little error.

[Brief explanation of the drawing]

1 is a diagram showing the configuration of an embodiment of an image processing device according to the present invention, FIG. 2 is a diagram illustrating the internal configuration of a moving vehicle used in the embodiment, and FIGS. 3A and 3B are 8 Figures 4A and 4B are diagrams explaining the principle of Lagranche interpolation, Figure 5 is a flowchart showing the procedure related to control of the embodiment, and Figure 6 is a diagram used for prediction. FIG. 7 is a diagram illustrating the drawbacks of the conventional method. This is the road guidance section. In the figure, 1... moving vehicle, 2... camera, 3... encoder sensor, 4... vehicle controller, 5... traveling controller, 6... camera angle sensor, 10... image Processing device, 10a... image extraction unit, fob... prediction unit, 10c... correction unit, 10d... interpolation formula generation unit,
11... Figure 4A Figure 4B

Claims (1)

[Claims] (1) In a moving vehicle that performs driving control based on the recognized road edge, an imaging means takes an image of the road to recognize the outside world, and a vehicle detects the road edge based on a plurality of recognition results regarding the road edge. An image processing device for a moving vehicle, comprising: means for extracting an image processing target area for recognition from the captured image; and means for recognizing a road edge from information on the extracted image area.