JP2728171B2 - Mobile car image processing device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to an image processing apparatus for a moving vehicle for recognizing a road end from an image of the road.

2. Description of the Related Art Generally, a mobile vehicle that performs autonomous traveling control by recognizing the outside world detects a roadside or the like and performs traveling control along the roadside. For example, JP-A-61-240307
As shown in the figure, an image is fetched, the infinity point on the roadside is recognized from the image, and the traveling control of the mobile vehicle is performed based on the infinity line defined from this point. In other words, for traveling control, it is necessary to know at least an error of the lateral displacement (distance from the traveling road end) with respect to the target course.

(Problems to be Solved by the Invention) By the way, in order to recognize the roadside edge from the image, as shown in FIG. 7, the linear equation y = kx
It is necessary to find + l. This straight line is recognized as being in contact with the road end. However, conventionally, image data of one entire screen is processed for obtaining this equation, and it takes a long time to recognize a road end, and is not suitable for high-speed processing.

Therefore, the present invention has been proposed in order to eliminate the drawbacks of the above-described conventional example. The purpose of the present invention is to narrow down the image processing target area in some form and perform image processing for roadway edge recognition at high speed. The present invention proposes an image processing device for a mobile vehicle that can perform the image processing.

(Means and Actions for Achieving the Object) A configuration of an image processing device for a mobile vehicle according to the present invention for achieving the above object is provided in a mobile vehicle that performs travel control based on a recognized travel road edge. Imaging means for taking an image of the traveling road for recognition, and means for extracting an image processing target area for traveling road edge recognition from the captured image based on a plurality of recognition results regarding the traveling road edge, Means for recognizing the road end from the information of the extracted image area.

(Embodiment) With reference to the accompanying drawings, an image processing apparatus of the present invention will be described.
An embodiment applied to recognition of a road end for traveling control will be described.

(Principle of Embodiment) This embodiment focuses on the fact that the structure of a road changes continuously and smoothly, especially in an environment such as a highway. That is, from the information of the traveling road edge recognized in the past, a likely range of the present or future traveling road edge is predicted,
The predicted range in the image obtained at the present time is set as a search range for roadway edge recognition. If the road structure changes continuously and smoothly, the possibility that the road end lies within such a predicted range is considerably increased. And
If image processing is concentrated within the narrowed range, the speed of roadside recognition can be improved.

The equation of the straight line representing the road end can be basically defined by two parameters. These parameters are, for example, in the: xy rectangular coordinate system, a slope and an intercept (in the case of FIG. 7, a slope k and an intercept l) or coordinate positions of two points. When a point on the straight line is given, the distance ρ between the point and a predetermined origin, and the intersection angle θ between a line connecting the origin and the point and the coordinate axis.

There are various other methods of expressing a straight line. As described above, since the straight line in the plane is defined by the two parameters, in this embodiment, the parameters of the straight line recognized in the past are extrapolated to the present or future, and the approximate parameters (having a range) are obtained. , And corrects the image obtained at this time using the predicted parameters as initial information.
That is, the image area is narrowed down. Then, a straight line representing the road end is obtained at high speed and accurately from the narrowed-down images. That is, in this embodiment, (i): prediction of parameters based on past recognition results, (ii): setting of a search range based on prediction, (iii): image extraction based on the search range, (iv) extraction The roadside recognition is performed in a cycle of calculating parameters based on the image data of the part, and (v): updating the recognition result.

<Configuration of Example Apparatus> FIG. 2 shows a mobile vehicle 1 when the image processing apparatus according to the present invention is incorporated in a mobile vehicle for performing travel control.
FIG. 3 is a diagram showing the configuration of FIG. The moving vehicle 1 includes a video camera 2 for taking an image of the outside world, a traveling control controller 5 incorporating an image processing device, an encoder sensor 3 for detecting the number of rotations of wheels for measuring a traveling distance, and an actual moving vehicle. 1 is mounted with a vehicle controller 4 for controlling the traveling of the vehicle. The mobile vehicle 1 is driven by a normal gasoline engine (not shown) and is steered by a steering mechanism (not shown). The vehicle controller 4 controls the traveling speed of the moving vehicle by controlling the throttle actuator, controls the traveling direction of the moving vehicle by controlling the steering actuator, and controls the brake actuator (not shown). Is controlled so that the moving vehicle is braked.
Reference numeral 6 denotes a sensor for detecting the mounting angle (α) of the camera 2, and estimates the horizontal line in the image based on the angle and the camera mounting position d.

FIG. 1 shows details of the travel control controller 5. The controller 5 includes an image processing unit 10 and a route guidance unit 11, as shown in FIG. The image processing unit 10 includes an image extraction unit
10a, a prediction unit 10b, a correction unit 10c, and an interpolation formula generation unit 10d. According to the interpolation formula generated by the interpolation formula generation unit 10d (which is generated based on past data), the prediction unit 10b determines parameters ρ _P and θ _P that will define the road end of the current image. (Ie, predict). This prediction method follows the Lagrange interpolation method described later. The prediction unit 10b modifies the prediction parameter ρ _P θ _P
Output to 0c. The correction unit 10c outputs data [R, Θ] regarding the search range in the image to the image extraction unit 10a based on the parameters ρ _P and θ _P. The image extracting unit 10a sends the image data of the extracted area to the correcting unit 10c. The correction unit 10c outputs a parameter ρ _C θ _C that defines an approximate straight line representing a roadside by using a Hough transform described later. The road-edge information parameter [rho _C theta _C is sent to the route guidance unit, where the steering angle for performing travel control based on the error between the automatic position and Michitan, by calculating the vehicle speed, the vehicle controller 4 Output.

In this embodiment, a method is employed in which a straight line equation defining a road end is represented by the above-mentioned parameters ρ and θ.
This is mainly because the parameters ρ and θ are easy to use for the Hough transform. Also, the Hough transform can be performed faster to obtain a linear equation than the least squares method.

<Hough Transform> The Hough transform will be described with reference to FIG. The roadside is a group of “1” pixels when input as an image. Hough transform is performed when the pixel points of "1" are linearly distributed.
In this method, a straight line along each point is approximately obtained.

In XY space, as in the Figure 3A, two _{points, A: (X A y A} ) B: a is (x _B y _B). These points represent the pixel of "1". Any straight line passing through the point A is represented by ρ = X _A cos θ + y _A sin θ, and any straight line passing through the point _B is represented by ρ = x _B cos θ + y _B sin θ. If ρ and θ are considered as variables, a straight line group passing through the point A is like a triangular function curve I in FIG. 3B, and a straight line group passing through the point B is like a curve II in FIG. 3B. .
Ρ _P and θ _P representing a straight line passing through points A and B are represented by curves I and II
Should be at the intersection of That, [rho _P, with respect to _{θ P, ρ P = x A} cosθ P + y A sinθ P ρ P = x B cosθ P + y B sinθ P holds. A straight line passing through two points is uniquely determined, but a straight line passing through three points is generally determined only approximately. Three ρθ curves corresponding to respective straight line groups passing through three points intersect at three points. If a straight line approximation is possible so that these three points can be placed on substantially one straight line, the three intersection points should be close to each other. By developing this idea, the group of “1” pixels in the image can be converted into a ρθ curve on the ρθ plane corresponding to each of the straight lines passing through each pixel. The specific [rho ^* in the distribution of the intersection of these curves between a straight line is Yotsute identified theta ^*, for ^{_{example, ρ * = x A cosθ *}} + y A sinθ * in Yotsute, pixel groups are linearly approximated You.

However, it is not realistic to perform such a process of obtaining an approximate straight line by the Hough transform for all points in an image. Because there is an image other than the roadside in the image, the approximate straight line passing through all the pixel points including the pixel unrelated to the roadside is calculated, and the straight line approximation of the roadside is calculated. This is because doing it becomes meaningless.

Therefore, in the present embodiment, the process of obtaining an approximate straight line by the Hough transform is performed as follows. Obtained i.e. accordance connexion to prediction based on interpolation formula below, past based on the road-edge information, the predicted value will get there are road-edge of current [rho _P, theta _P a (subscript P is to mean prediction) On the basis of the predicted value, a certain range is set as a search target range of the roadside. In the present embodiment, the interpolation expression follows the Lagrangian interpolation method as an example. Also, the above-mentioned fixed range is
set of ρθ [R, Θ] is defined _{by, R∈ [ρ P -Δρ, ρ} P + Δρ] Θ∈ [θ P -Δθ, θ P + Δθ] is in the range of. With such a prediction, the Hough transform is limited to the range III as shown in FIG. 3B, and the straight line extraction processing is accelerated.

<Lagrangian interpolation method> A Lagrange interpolation method will be described with reference to FIG. This interpolation method is a method of interpolating the next data based on the past n data, and is represented by an interpolation polynomial at the time of n.

FIG. 4A and FIG. 4B show that the moving vehicle is traveling at a position d ₀ d ₁ d ₂
.., D _n , which represent changes in the parameter ρθ obtained in the past. For example, a roadside straight line (ρ ₀ θ ₀ ) is obtained when the vehicle is at the position d ₀ , and a road end straight line (ρ ₁ θ ₁ ) is obtained when the vehicle is at the position d _{1. n}
θ _n ) was obtained. Then, by predicting the ρθ when you are from these data the running position d, ρ _P (d), and to obtain theta _P a (d). Note that d is the encoder sensor 3
Obtained from

Since the prediction method for ρ _P and θ _P is the same, the prediction for ρ will be described below.

In FIG. 4A, the points (d ₀ ρ ₀ ), (d ₁ ρ ₁ ) ‥‥‥,
The ( _n ) th order interpolation polynomial that passes through (d _n ρ _n ) and gives the predicted value ρ _P (d) is Is represented by Here, the coefficients of the (n + 1) number of a ₀ ~a _n This polynomial. The points (d ₀ ρ ₀ ), (d ₁ ρ ₁ ), ‥‥‥,
Since (d _n ρ _n ) is a point in the past, their values are known. Also, ρ of the general formula given by the above formula
_{Since P} (d) passes through n + 1 points, By solving the n + 1 equations, the coefficient a _0
If you ask ~ an Therefore, ρ _P (d) when predicted at point d is It is expressed as θ _P (d) is obtained in exactly the same way. For the sake of distinction, if the coefficient for θ _P (d) is represented by b, And θ _P (d) when predicted at point d is It is.

In other words, historical data _{d 0 ~d n, a 0 ~a} n, b 0 ~b n, ρ 0 ~
If ρ _n and θ _{0 to} θ _n and the current position d are known, ρ
_P (d) and θ _P (d) can be predicted.
Let _P (d) and θ _P (d) be ρ _P θ _P , and the search range [R, Θ], R ∈ [ρ _P −Δρ, ρ _P + Δρ] Θ∈ [θ _P −Δθ, θ _P + Δθ] ] Is used for the determination.

<Control Procedure> FIG. 5 is a flowchart showing a processing control procedure in the image processing unit 10. In step S2, the traveling distance d is obtained from the encoder 3. In step S4, past data
d ₀ ~d _n, reads _{a 0 ~a n, b 0 ~b} n, ρ 0 ~ρ n, the θ ₀ ~θ _n from memory. These past data are stored in a predetermined memory so that the index can be read out, as shown in FIG. In step S6, the interpolation polynomial ρ _P (d), θ
_P (d) is generated, and in step S8, new coefficients a 'and b'
(Such as a ₁ ′ in FIG. 6) and calculate ρ based on the above-described equation.
Calculate _P (d) and θ _P (d). Step S10 in the search range [R, Θ], R∈ [ ρ P -Δρ, ρ P + Δρ] Θ∈ [θ P -Δθ, θ P + Δθ] determined. At this time, Δρ and Δθ may be selected, for example, as follows: Δρ = | ρ _P −ρ ₀ | / 2 Δθ = | θ _P −θ ₀ | / 2 This is because ρ ₀ and θ ₀ are past data used for prediction and do not become values far from predicted values.

In step S12, this search range is sent to the extraction unit 10a, and image data in that range is cut out.
In S14, based on the image data, an approximate straight line ρ _C = x _i cos θ _C + y _i sin θ _c is calculated according to the Hough transform described above. In step S16, ρ _C and θ _C are sent to the vehicle controller.

In step S16, for the next prediction, the currently calculated a ', b', are moved to the area a, b and updated (see FIG. 6). Similarly, ρ _n ← ρ _C θ _n ← θ _C is also updated.

The above control procedure is repeated every time one image is input. Thus, the image processing area is narrowed down by performing the prediction, and the parameter defining the straight line representing the roadside can be calculated at high speed. Then, traveling control can be performed at high speed from the road edge recognized at high speed.

In the above description, the interpolation polynomial is of order n, but it is appropriate that n = about 3 to 5 from the balance between prediction accuracy and calculation time. Further, Δρ and Δθ may be determined according to the prediction accuracy in addition to the above-described definitions.

In the above embodiment, the Hough transform is used for the straight line extraction. However, this is used because of the calculation speed, and the edge detection difference method can be applied. In addition, a Newton interpolation polynomial may be used as the interpolation formula in addition to the Lagrangian polynomial. That is, in the present invention, a search range for extracting a straight line is predicted from past data, and a straight line extraction method, a straight line extraction method, and a prediction method are not limited as long as a straight line is extracted within this range. is there.

(Effects of the Invention) As described above, the configuration of the image processing device for a moving vehicle according to the present invention is applied to a moving vehicle that performs running control based on a recognized running road edge. Imaging means for photographing, based on a plurality of recognition results related to the roadside, an image processing target area for the roadside recognition,
It is characterized by comprising means for extracting from the captured image, and means for recognizing the road end from the information of the extracted image area.

Therefore, since the image processing target area is narrowed down,
The time for roadside recognition is reduced. Further, since the image processing target area is defined based on the past recognition result, even if such a narrowing is performed, an error is small.

According to a preferred aspect of the present invention, the accuracy and speed of roadside recognition are improved by linearly approximating the roadside.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention, FIG. 2 is a diagram for explaining an internal configuration of a moving vehicle used in the embodiment, FIG. 3A and FIG. 4A and 4B are diagrams illustrating the principle of Lagrangian interpolation, FIG. 5 is a flowchart illustrating a procedure related to control of the embodiment, and FIG. 6 is used for prediction. FIG. 7 is a diagram showing a configuration for storing past recognition result data, and FIG. 7 is a diagram for explaining a conventional defect. In the figure, 1 ... moving vehicle, 2 ... camera, 3 ... encoder sensor, 4 ...
Vehicle controller, 5: Travel controller, 6: Camera angle sensor, 10: Image processing device, 10a: Image extraction unit, 10b
.., A prediction unit, 10c, a correction unit, 10d, an interpolation formula generation unit, and 11, a route guidance unit.

Claims (2)

(57) [Claims] An image pickup means for taking an image of a traveling road for recognizing the outside world, and traveling based on a plurality of recognition results relating to the traveling road edge. An image processing device for a mobile vehicle, comprising: means for extracting an image processing target area for roadside recognition from the captured image; and means for recognizing a roadside edge from information on the extracted image area. 2. The apparatus according to claim 1, wherein said recognizing means recognizes a road end from the image area by linear approximation.